What changes when the system stops advising and starts acting

Reversibility, not accuracy, is the governing design variable. This paper sets out why, and what follows from taking it seriously

1. Executive summary

The preceding sequence of papers ended by drawing a line. A knowledge-grounded system produces evidence; a competent person decides; the engineering authority acts. Nothing machine-initiated crosses into actuation. That containment argument is what made the system assurable, and it remains correct for the diagnostic and explanatory work it was built for.

This series concerns what happens when something must cross the line. Not because the containment argument was wrong, but because a large class of industrial work consists of actions that are individually low-consequence, high-volume, and entirely tractable — issuing a request for quotation, updating a delivery schedule, opening a notification, chasing a document, reconciling a discrepancy. Requiring a human commitment for every one of these caps the value of the system at the availability of human attention, which is the scarcest resource in the organisation and the one that degrades fastest under repetition.

The temptation is to treat this as a capability question: as models improve, allow them to do more. That framing is wrong and it produces bad systems. The shift from advising to acting is not an increment along a scale of competence. It is a change in the nature of the failure. An advisory system that is wrong produces a false belief in a human who may or may not catch it. An acting system that is wrong produces a change in the state of the world, which persists whether or not anyone catches it.

Once that distinction is taken seriously, the governing design variable is not accuracy but reversibility. This paper argues that agentic system design consists almost entirely of identifying the moments at which the world changes irreversibly — the commitment points — and arranging everything else around them. It develops a classification of actions by cost to undo and blast radius, examines why the familiar "human in the loop" formulation is not a control unless the loop is placed correctly, and sets out what carries over from the assurance framework of the preceding paper and what does not.

2. Why the line has to move

2.1 The cost of full containment

Containment is not free. Under a strict advisory architecture, every unit of value delivered requires a unit of human attention to authorise it. The system's throughput is therefore bounded by reviewer capacity, and it does not improve as the system improves — a better model produces better recommendations, but each still needs the same review.

For diagnostic work this is the right trade. A recurring failure investigation happens a handful of times a year, the consequence of error is large, and the review is a small fraction of the total effort. For transactional work the arithmetic inverts. A procurement function handling several hundred quotation cycles a month cannot justify a considered human commitment on each; the review becomes perfunctory, and a perfunctory review is not a control. It is a control-shaped object that generates an audit record while providing no actual assurance.

This is the honest case for agentic systems in industrial operations, and it is worth stating in these terms rather than in terms of autonomy or capability. The argument is not that the machine should be trusted more. It is that human attention should be concentrated where it changes the outcome, and that spreading it thinly across hundreds of reversible transactions is a way of appearing to exercise control while actually abandoning it.

2.2 What actually changes

The difference between an advisory and an acting system is not degree. Four things change discontinuously at the boundary.

•    Errors persist. A wrong recommendation is discarded when noticed. A wrong action has already changed the world; noticing it initiates a recovery, not a correction.

•    Errors compound. An acting system typically acts again on the state it produced. A single early mistake propagates through subsequent steps, each of which is individually reasonable given the state it observed.

•    The observer disappears. The advisory architecture guarantees that a human reads every output. Once actions are taken without review, the majority of the system's behaviour is never seen by anyone. Failures are discovered through their consequences rather than through inspection.

•    Third parties become involved. Many industrial actions are visible outside the organisation — a message to a supplier, a booking, a submission to a regulator. These cannot be unilaterally reversed, because someone else has already responded to them.

The fourth is the one most often underestimated. An internal record can usually be corrected. A quotation request that reached forty suppliers cannot be recalled, and the reputational and commercial consequences of retracting it are real even when the operational consequences are trivial.

2.3 The framing to avoid

Much current discussion organises agentic systems along a scale of autonomy borrowed loosely from vehicle automation: assisted, supervised, conditional, full. The analogy is appealing and unhelpful here, for two reasons.

First, it describes how much the machine does rather than what the machine does to the world, and the latter is what determines risk. A system operating at high autonomy over purely reversible actions is far safer than one operating under close supervision over irreversible ones. An autonomy level conveys nothing about which of these you have.

Second, it implies a progression — that an organisation should advance through the levels as confidence grows. In practice the correct arrangement is not a level at all. It is a mixed regime in which a single workflow runs autonomously across dozens of reversible steps and requires explicit commitment at one or two specific points, permanently, regardless of how good the system becomes. The commitment point does not move as the model improves, because its position is determined by the consequence of the action, not by the reliability of the actor.

3. Classifying actions

A useful classification of actions must discriminate along the dimensions that determine what can go wrong. Two do most of the work.

3.1 Cost to undo

Actions divide into three classes by what recovery requires.

•    Reversible. The prior state can be restored by the system itself, with no residual trace beyond an audit entry. A draft is revised, a forecast is recomputed, a staged change is discarded. Recovery is a system operation.

•    Compensable. The prior state cannot be restored, but a further action can offset the effect at a bounded and acceptable cost. A requisition is cancelled, a booking is released within its free window, a spare is returned to stores. Recovery is a business operation with a real but tolerable price.

•    Irreversible. No action restores or offsets the effect. A contract is accepted, a ledger period is closed, a regulatory submission is made, a machine is tripped, a record subject to retention rules is deleted. Recovery does not exist; there is only consequence management.

Two cautions on this classification. Reversibility is a property of the action in its context, not of the operation in the abstract — cancelling a purchase order is compensable on day one and effectively irreversible once the supplier has begun manufacture. And an action that is technically reversible may be practically irreversible because someone has already observed it and acted, which is why anything visible to a third party should be treated at least one class more severely than its mechanics suggest.

3.2 Blast radius

The second dimension is the scope of state affected: a single record, a process unit, enterprise-wide state, or a party outside the organisation. Blast radius matters independently of reversibility because it determines how much has to be examined when something goes wrong, and how many other parties are entitled to an explanation.

Blast radius must be assessed for the chain, not the step. An individually narrow action repeated two hundred times before anyone notices has an enterprise-scale radius, and this is the characteristic failure of automated systems: not one catastrophic action but a defensible small action executed consistently on a wrong premise. Rate limits and cumulative-effect thresholds are therefore not operational conveniences. They are the mechanism by which blast radius is bounded at all.

3.3 Reading the grid

The frontier in Figure 1 is drawn where it is for a reason worth making explicit: it does not follow the diagonal. Irreversible actions require commitment at every blast radius, including the narrowest — deleting a single retained record is beyond the frontier while adjusting a process set point may not be. Compensable actions require commitment only once their radius reaches enterprise or external scope, where the compensation itself becomes visible and costly.

This shape captures the paper's central claim. An organisation that places its controls according to how likely the system is to err will gate the difficult judgements and wave through the easy irreversible ones. An organisation that places them according to cost to undo will do the opposite, and will be right.

4. The commitment point

Every agentic workflow contains one or more moments at which the world changes in a way that cannot be walked back. Identifying these, and treating them as the organising feature of the design, is most of what distinguishes a well-built acting system from a poorly-built one.

4.1 Everything before, everything after

A typical workflow decomposes into retrieval, reasoning, drafting, staging, commitment and reconciliation. The first four are reversible: they produce artefacts and proposed state, all of which can be discarded at no cost beyond wasted computation. The fifth changes the world. The sixth confirms that the change took effect as intended.

This asymmetry is the design lever. Because everything before the commitment point is reversible, the system can be given wide latitude there — it can explore, revise, discard, retry, and pursue several approaches in parallel, all without any control being necessary and without any human attention being consumed. The entire control budget can then be spent on the one moment that matters.

4.2 Why distributed approval fails

Requiring approval at each step appears more careful and is in fact substantially less safe. Three mechanisms drive this.

Approval fatigue is the most familiar: a reviewer presented with a hundred low-stakes confirmations develops the habit of approving, and carries that habit into the one confirmation that mattered. This is not a training failure. It is the predictable result of a design that spends the reviewer's attention on items where it makes no difference.

Context fragmentation is less discussed and equally damaging. A reviewer approving a drafting step sees a draft; they do not see what will ultimately be committed, because it does not exist yet. Approval given at an early step is therefore approval of an intention, not of an outcome, and it produces an audit record that overstates what was actually reviewed.

Responsibility diffusion completes the picture. When four people have approved four steps, no one has authorised the action. Each can point to the narrowness of what they saw. A single commitment gate, by contrast, places the whole action in front of one identified person who is unambiguously accountable for it.

4.3 What the gate must show

A commitment gate is only a control if what it presents is sufficient to decide on. The minimum content follows from the preceding paper's evidence-package model, extended for action:

•    The action in concrete terms — what will change, in which system, affecting which records or parties

•    The reasoning and the evidence behind it, with citations and graph paths, as for any advisory output

•    The reversibility class and, where the action is compensable, the specific compensating action and its cost

•    The blast radius, including the cumulative effect of this action together with related actions already taken in this run

•    What the system considered and rejected, so the reviewer sees that alternatives existed

•    The confidence, and explicitly whether this action falls inside the qualified domain

An approval interface that shows only a summary and two buttons is a mechanism for collecting consent, not for exercising judgement, and it should not be described as a control in any assurance argument.

4.4 Moving the point deliberately

The commitment point can be moved, but only by changing the action rather than the confidence. Useful moves include converting a commitment into a staged change that a downstream process confirms; batching many small commitments into one reviewable package; introducing a delay window during which the system's own reconciliation may catch an error before the effect becomes visible; and splitting an action so that the reversible majority proceeds automatically while only the irreversible remainder is gated.

Each of these is a design intervention with a cost. None of them is achieved by deciding that the system is now reliable enough. 

5. Action tiering

The preceding paper introduced decision tiers, assigned by the consequence of the decision the output informs. Acting systems require a second, orthogonal classification: action tiers, assigned by what the action does to the world. The two are independent — a low-tier decision can trigger a high-tier action, and this combination is where unexamined risk accumulates.

5.1 Standing authorisation and its limits

The A3 row introduces the only defensible mechanism for reducing human involvement without reducing control: a standing authorisation. A named accountable person authorises a class of action, within stated parameters, for a stated period, having reviewed the class rather than each instance. This is ordinary industrial practice — it is how delegated financial authority and permit-to-work systems already operate — and it transfers cleanly.

It works only if four conditions hold. The class must be defined by properties the system can evaluate reliably, not by a vague description. The parameters must include quantitative limits on value, volume and rate. The authorisation must expire and require renewal rather than persisting indefinitely. And the authoriser must receive a periodic summary of what was done under it, in a form that supports withdrawal.

The fourth condition is the one most commonly omitted, and without it a standing authorisation is indistinguishable from an absence of control that happens to have a signature attached.

5.2 Tier assignment is a design output

Tiers are assigned to actions during design, recorded in the system, and enforced mechanically at the point of execution. They are not a policy statement in a governance document. If the system can execute an A4 action without encountering a gate, the tier is a description of intent rather than a control, and the distinction will only be discovered when it matters.

6. What carries over, and what does not

The assurance framework of the preceding paper was built for a system that produces evidence. Much of it transfers to a system that acts; some of it does not, and the parts that do not are where new work is required.

6.1 Inaction is also an action

An advisory system that abstains produces no effect. An acting system that abstains may produce a substantial one: a quotation not issued before a deadline, a shipment not booked, a discrepancy not escalated. In a workflow the system has taken responsibility for, silence is a decision with consequences.

The design obligation is that inaction must be safe and must be visible. Every abstention should generate a human-visible item with a deadline, and the workflow should have a defined behaviour when the system stops rather than an implicit assumption that someone will notice. Systems that fail silently in an automated pipeline are a well-understood hazard in process control; the same discipline applies here and is frequently forgotten because the failure looks like nothing happening.

6.2 Detection moves from review to monitoring

The most significant structural change is that inspection no longer catches errors, because most actions are never inspected. Detection must come from monitoring the effects: reconciliation between what the system believes it did and what the target systems record, distributional monitoring of action volumes and values against expectation, and downstream signals such as supplier queries, rejected transactions and unexpected reversals.

This is a different discipline from evaluation and requires different instrumentation. It is the subject of the fourth paper in this series, but it should be designed in from the beginning, because a system built without it cannot have monitoring added afterwards in any meaningful sense — the necessary reconciliation points have to exist in the architecture.

7. A worked boundary: procurement

To make the framework concrete, consider a quotation-to-award workflow of the kind common in petrochemical and heavy engineering procurement. The system receives a requirement, identifies candidate suppliers, issues requests for quotation, receives and normalises responses, evaluates them against stated criteria, and produces a recommendation.

Applying the classification produces a specific and defensible arrangement rather than a general disposition toward caution.

Two features of this arrangement are worth drawing out. The overwhelming majority of the work — and of the elapsed time — runs without human involvement, because the overwhelming majority of the steps are reversible. And the single gate sits at the award, where it would sit in a well-run manual process, for exactly the same reason.

The step that repays the most thought is issuing the RFQ. It is externally visible and cannot be recalled, which argues for a gate. But its actual consequence is modest, its volume is high, and its parameters are enumerable — supplier list, value band, commodity class. It is therefore the natural candidate for standing authorisation with limits, and the design question becomes what the limits are and who reviews the summary, rather than whether a human clicks approve four hundred times a month.

The supplier query step illustrates a further technique. Free-text correspondence with a supplier is unbounded in what it might commit the organisation to; a constrained set of query templates is bounded and can be assessed once. Restricting the action rather than reviewing each instance converts an unassessable action into an assessable one, which is the same move as re-engineering an irreversible action into a compensable one and is available far more often than it is used.

8. How to start

Step 1 — Enumerate the actions, not the use case

Before any design, list every action the system will be capable of taking, in concrete terms — the specific operation on the specific system. Vague scoping such as "manages the RFQ process" hides the actions and therefore hides the risk. The list is usually shorter than expected and is the single most useful artefact in the programme.

Step 2 — Classify and place the frontier

Assign each action a reversibility class and blast radius, and assign a tier. Where classification is contested, that disagreement is a finding and should be resolved by the accountable business owner rather than the build team.

Step 3 — Re-engineer before you control

For every action landing beyond the frontier, ask whether it can be moved: soft rather than hard, provisional rather than confirmed, staged rather than committed, templated rather than free-form. Each successful move removes a gate permanently. This step is routinely skipped, and it is the highest-return step in the sequence.

Step 4 — Design the gate, then the automation

Specify what the commitment gate presents and who is accountable for it before building the autonomous portion. A gate designed afterwards is invariably a summary and two buttons, because by then the information needed to decide well was never retained.

Step 5 — Bound the chain

Set rate limits, cumulative value limits and run-level caps. These bound blast radius for the repeated-small-action failure, which is the mode most likely to occur and least likely to be anticipated.

Step 6 — Build reconciliation from the outset

Every action must be followed by a check that the target system reflects what was intended. Discrepancy between intent and effect is the primary detection mechanism once review is no longer universal, and it cannot be retrofitted.

Step 7 — Define what happens when it stops

Specify the behaviour on abstention, on error, and on the system being unavailable. Assign the resulting queue to a named owner. A workflow whose failure mode is undefined has an actual failure mode of nothing happening quietly.

9. Closing observations

The move from advising to acting is usually discussed as a question of trust, and framed as something to be earned incrementally as systems improve. That framing produces a predictable pattern: capability improves, confidence rises, controls are relaxed uniformly, and the first serious incident involves an action that was never risky because it was hard, but because it could not be undone.

The alternative framing is structural. Actions are classified by what they do to the world. Those that can be undone need no gate regardless of how uncertain the system is, because the cost of being wrong is a retry. Those that cannot be undone need a gate regardless of how reliable the system has become, because the cost of being wrong is permanent. The frontier between them is a property of the work, not of the technology, and it does not move when the model improves.

What does move, and what genuinely repays engineering effort, is the set of actions that must be irreversible at all. Most systems inherit far more irreversibility than the underlying business requires — hard deletes where soft ones would serve, confirmed bookings where provisional ones would, free-text correspondence where templates would. Every one of those converted is a control removed permanently rather than a control exercised repeatedly.

Design the commitment points. Automate everything between them without apology.

About this paper

This is the first paper in "Systems That Act", a four-part series following "Knowledge Graphs for the Industrial Enterprise", "Beyond Either" and "Fit to Decide". Subsequent papers address architecture, multi-agent and inter-organisational operation, and assurance for systems that change state. The procurement example is illustrative and generic; it does not describe any specific client implementation. The paper is vendor-neutral: capabilities are described functionally and no specific platform, model or agent framework is recommended.