The Tool Surface

Share
The Tool Surface

Architecture for systems that change the state of the world

An agent’s safety is determined by what it is able to express, not by what it has been told. This paper is about building the part that cannot be talked out of.

1. Executive summary

The first paper in this series argued that agentic system design consists of identifying the moments at which the world changes irreversibly, and arranging everything else around them. This paper is about the machinery that makes such an arrangement enforceable rather than aspirational.

Its central claim is that an acting system's safety properties are determined by its tool surface — the set of operations it is able to perform and the shape of their parameters — and not by the model, the prompt, or the policies written about it. An instruction to the model is a request. A tool that cannot express a dangerous operation is a control. Almost every serious failure in a deployed agentic system traces back to the gap between those two things: an organisation believed it had constrained behaviour when it had only described the constraint.

From that claim follows most of the architecture. The planner proposes; a deterministic policy engine outside the model validates what it proposed, as a whole plan rather than a step at a time; a typed tool surface with explicit preconditions executes only what the policy engine permits; an execution engine enforces idempotency, ordering and cumulative limits; a compensation registry holds a tested reverse action for every offsettable step; and reconciliation compares what the system intended against what the target systems actually record. The knowledge graph of the earlier series becomes the agent's model of the world, which is what allows preconditions to be checked against reality rather than against the model's belief.

Three further topics receive extended treatment because they are where industrial deployments fail in practice: ordering plans so that irreversible steps come last, which converts abort capability from an accident into a design output; the injection surface created by agents that read supplier correspondence and vendor documents, where content is routinely mistaken for instruction; and the observability that must be built in from the start, because once most actions are never reviewed, detection depends entirely on reconciliation.

 

What this paper assumes

Familiarity with the reversibility classification and action tiers of paper one, and with the governed knowledge graph and evidence-package model of the preceding series. Where those disciplines are absent, the architecture here can still be built, but several of its controls — precondition checking, temporal scoping, write-back governance — have nothing to rest on.

2. The tool surface is the safety architecture

2.1 Instructions are not controls

It is common to see safety requirements expressed as instructions: never issue a purchase order above a stated value, always confirm before contacting a supplier, do not modify records outside the current work order. These are useful. They are not controls, because compliance is a behaviour of a probabilistic system rather than a property of the architecture.

The test is simple and worth applying to any agentic design: if the model were to attempt the prohibited action, what would physically prevent it? If the answer is that the model would not attempt it, there is no control. If the answer is that the attempt would be rejected by a component that does not consult the model, there is one.

This distinction is not a comment on model reliability. It is the same principle that underlies every other safety-relevant system in an industrial facility: a relief valve is not a procedure telling the operator to reduce pressure. Controls are things that hold when the actor misbehaves, and the misbehaviour need not be malicious or even mistaken — an unusual input, an ambiguous instruction or an adversarial document is sufficient.

2.2 Design tools so that the dangerous action is inexpressible

The strongest available lever is the shape of the tool itself. A tool that accepts a free-form database query can express any operation the connection permits, and no amount of instruction narrows that. A tool that accepts a stock reservation for a specified item, quantity and location, with the quantity bounded and the location drawn from an enumerated set, can express reservations and nothing else.

The design principle is to make the tool as narrow as the work requires and no narrower, and to encode the constraints in the parameter types rather than in guidance:

•    Enumerated parameters wherever the set of valid values is known — locations, suppliers on an approved list, document types, notification templates

•    Bounded numerics with limits enforced by the tool, not stated in its description

•    No free-text parameter that reaches an external party without passing a gate; templated correspondence with typed slots instead

•    Separate read and write tools, so that read-only workflows cannot mutate state even in principle

•    Separate tools for reversible and irreversible variants of the same operation — a soft-delete tool and a hard-delete tool are different tools with different tiers, not one tool with a flag

The last point generalises. Every boolean flag on a tool that changes its reversibility class is a place where a single token determines whether an action can be undone. Splitting such a tool costs nothing and removes the failure mode entirely.

2.3 Preconditions, postconditions and invariants

Each tool should declare what must be true before it runs, what will be true after, and what must remain true throughout. These declarations are executed, not documented.

•    Preconditions are checked against the world model and the target system immediately before execution — not against the state the planner assumed when the plan was made, which may be minutes or hours stale. A reservation tool checks that the stock still exists; an award tool checks that the quotation has not expired.

•    Postconditions are verified after execution and are the basis of reconciliation. A tool that reports success without verifying its postcondition has reported that a request was accepted, which is a different fact.

•    Invariants are properties of the whole run: total committed value, number of external contacts, count of records modified. They are checked continuously and breach halts the run rather than the step.

Precondition failure is not an error to be worked around. It is the signal that the world moved and the plan is stale, and the correct response is to return to planning, not to retry.

2.4 Idempotency and the duplicate action problem

An underappreciated failure mode, and one that occurs early in most deployments: a tool call times out, the caller does not know whether the operation completed, and a retry produces a second purchase order, a second notification, a second stock movement. The target system has no way to know the two requests were meant to be one.

Every state-changing tool must therefore accept a caller-generated idempotency key, and the execution engine must generate that key deterministically from the plan step rather than randomly per attempt. Where the target system does not support idempotency keys — which describes a great many enterprise systems — the execution engine must implement the equivalent itself: record the intent before attempting, query for the effect before retrying, and never retry blind.

This is unglamorous plumbing and it is the difference between a system that occasionally does something twice and one that does not. In a procurement or logistics context, doing something twice is a commercial event.

2.5 Granularity: no tool spans a commitment point

Tool granularity is a genuine design trade. Coarse tools produce short plans that are easy to validate but hide their internals; fine tools produce long plans that are transparent but brittle and slow.

One rule resolves most of the tension: no tool may span a commitment point. A tool that internally evaluates quotations and awards the contract has buried an irreversible action inside an operation whose signature does not reveal it. The policy engine cannot gate what it cannot see, and the tier assigned to the tool becomes a claim about its implementation rather than about its effect.

Where a coarse tool is genuinely wanted for efficiency, split it at the frontier: one tool performs everything up to the commitment and returns a staged result; a second, separately tiered, commits it.

3. Reference architecture

The components below are described functionally. What matters is not the product boundaries but which side of the deterministic line each capability sits on.

Figure 1 — Reference architecture. The planner proposes; only the deterministic column constitutes a control.

3.1 Plan, validate, execute

The separation of planning from execution is the structural counterpart of the tool-surface argument. The planner produces a declarative plan — an ordered set of typed tool invocations with their parameters, dependencies and expected effects — and that plan is an artefact that exists before anything happens.

This matters for a reason beyond inspectability. Blast radius is a property of the chain, not the step, and a plan is the only representation in which the chain is visible before it runs. A step-by-step agent that decides its next action after observing the last one can never be assessed as a whole, because the whole does not exist until it is finished. Total committed value, number of external parties contacted, and count of irreversible actions are all computable from a plan and none of them is computable from a step.

3.2 The policy engine

Validation is performed by a deterministic component that does not consult the model. It evaluates the plan against the action tiers of paper one and against configured limits, and returns one of three outcomes: permitted, permitted subject to a commitment gate at identified steps, or rejected with a reason.

Its checks are mechanical and enumerable:

•    Every step maps to a tool the system is authorised to use in this context, at a tier the current authorisation covers

•    Cumulative value, volume and external-contact counts fall within limits for the run and for the period

•    Irreversible steps are individually identified and gated, with no standing authorisation applied

•    Ordering constraints hold, including the irreversibility ordering of Section 4

•    Compensating actions exist and are currently available for every compensable step

•    Where the plan was influenced by untrusted content, the provenance restrictions of Section 6 apply

A rejected plan returns to the planner with the reason, which is more useful than a bare refusal and usually produces a compliant plan on the next attempt. Rejection counts by reason are a valuable operational signal: a rising rate of limit breaches usually means the workload has changed rather than that the model has.

3.3 Re-planning is not continuation

Reality diverges from plans. A quotation expires, a stock level changes, a supplier responds unexpectedly. When a precondition fails or an observation contradicts the plan's assumptions, the system must return to planning and the resulting plan must be revalidated in full.

The failure to enforce this is a common and consequential shortcut. A system that patches a plan mid-execution and carries on has escaped validation for everything downstream of the patch, and the cumulative limits computed against the original plan no longer describe what is being executed. Re-planning is cheap; escaping validation is not.

4. Ordering and recovery

4.1 Irreversibility ordering

Where the order of steps is not forced by dependency, it should be chosen so that reversible and compensable steps precede irreversible ones. The effect is to maximise the window during which the entire run can be abandoned at no cost.

Figure 2 — The same six steps ordered as reasoned, and reordered by reversibility. The abortable window is a design output.

This is a cheap intervention with a large effect and it is almost never applied, because plans are typically ordered by narrative logic — the sequence in which a person would describe the work — rather than by recoverability. A planner instructed to prefer this ordering, with the policy engine enforcing it where dependencies permit, converts every run into one that can be abandoned for as long as possible.

A related discipline: where several irreversible steps must occur, group them so that they execute as close together as possible, ideally behind a single commitment gate. Irreversible steps scattered through a long run create multiple points of no return and multiple partial-completion states, each of which requires its own recovery reasoning.

4.2 Compensation, not rollback

Transactions do not span enterprise systems. A run that has updated a maintenance system, notified a supplier and reserved stock cannot be rolled back; it can only be compensated by further actions that offset what was done. This is the long-established saga pattern, and its requirements are specific.

•    Every compensable action has a named, tested compensating action, defined at design time. Improvising compensation during an incident is how partial recovery becomes a second incident. If a compensating action cannot be written, the original action is not compensable and should be reclassified as irreversible.

•    Compensation runs in reverse order. Later effects are undone before earlier ones, because earlier state may be a precondition of the later compensation.

•    Compensating actions can themselves fail. The design must define what happens then: retry policy, escalation threshold, and the point at which the system stops attempting recovery and raises a human incident with a complete account of the state it has left behind.

•    Compensation is not silent. Anything externally visible that is compensated should generate a notification, because the counterparty has already seen the original.

The compensation registry should be exercised as part of routine testing rather than only when needed. A compensating action that has never been run is a plan, not a capability, and the discovery that it does not work will happen at the worst moment.

4.3 Partial completion is the normal state

A run that fails midway leaves the world in a state that matches neither the starting point nor the intended end. This is not an exceptional condition to be handled defensively; it is the ordinary outcome of any long-running process against systems that can be unavailable.

The architecture should therefore represent run state explicitly and durably — which steps were attempted, which were confirmed, which are of unknown outcome — and should be able to resume, compensate or escalate from any point. The unknown-outcome case is the important one, and it is where idempotency earns its cost: the system can safely establish what actually happened rather than guessing.

5. The world model

An acting system needs a representation of the state it is acting on. The governed knowledge graph described in the first series is that representation, and this is where the two series join.

5.1 What the graph provides

•    Identity. The agent must know that the tag in the work order, the material number in the procurement system and the item in the supplier quotation denote the same thing. Without a resolved identity layer, an agent acting across systems is acting on the assumption that names match, which they do not.

•    Preconditions grounded in reality. Checks run against the world model and the target system, not against what the planner inferred. The distinction matters most when it matters most — during a long run, under changing conditions.

•    History. The bitemporal record answers what was true when, which is what allows an agent to reason about whether a quotation was valid at the time it was evaluated rather than now.

•    Blast radius computation. Determining what else is affected by a proposed action requires traversal. The graph is what makes an impact assessment possible before the action rather than after it.

5.2 Staleness and concurrency

The agent's view of the world is always behind the world, and other actors — people, other systems, other agents — are changing state concurrently. Two disciplines follow.

First, read the critical state immediately before acting, not at planning time. The gap between plan and execution is where the world moves, and preconditions checked at planning time are assertions about the past.

Second, use compare-and-set semantics where the target system supports them: the action carries the version or state the agent believed and the target rejects it if that belief is stale. Where the target does not support this, the reconciliation step must detect the divergence afterwards, which is strictly worse but is the situation with most legacy systems.

5.3 Write-back and the record of intent

The graph records not only the state of the world but what the system attempted. Every action produces assertions: what was intended, under which plan, on what evidence, with what outcome, and by whose authorisation. These follow the write-back governance of the earlier series — provenance on every assertion, staged before promotion, retractable.

This record is the substrate for everything in the fourth paper. An acting system without a durable, queryable account of its own intents cannot be reconciled, cannot be audited, and cannot be investigated when it is wrong.

6. The injection surface

Industrial agents read documents produced outside the organisation: supplier correspondence, quotation responses, vendor field service reports, delivery notes, contractor submissions, free text in work orders. This is the point of the architecture — that material is where the knowledge is. It is also the point at which an acting system becomes attackable in a way an advisory one is not.

6.1 Content is not authority

Text arriving from outside the trust boundary may contain instructions. Whether placed deliberately by an adversary or accidentally by a supplier whose email footer happens to read as a directive, the model has no reliable means of distinguishing content it should reason about from content it should obey.

This cannot be solved by prompt engineering. Instructions to ignore instructions are themselves instructions, and every published attempt at delimiting untrusted content within a prompt has proven bypassable. The problem is architectural and the response must be architectural.

Figure 3 — Exposure arises where reading untrusted content, holding privileged access and acting externally coincide. Separating them is the control.

6.2 Separate the three capabilities

The practical response is to avoid combining, in one agent with one context, the ability to read untrusted content, access to privileged data or authority, and the ability to produce external effects. Several separations are available and they compose:

•    Extraction is not action. An agent that reads supplier correspondence should have no action tools at all. It emits structured, typed output — extracted quantities, prices, dates — which a separate acting component consumes as data.

•    Typed interfaces between stages. What passes from the extraction stage to the acting stage should be a schema-validated structure, not free text. A structure cannot carry an instruction.

•    Allowlisted targets. Recipients, endpoints and accounts are drawn from configuration, never from content. An agent must not be able to send anything to an address it read in a document.

•    Provenance-dependent tiering. Where the parameters of an action were derived from untrusted content, the action is tiered upward. An award whose supplier was determined by a document rather than by the approved vendor register requires commitment regardless of value.

6.3 The realistic threat in an industrial setting

It is worth being specific about what an attack looks like here, because the abstract framing invites dismissal. A supplier competing for an award has a direct commercial incentive and legitimate access to the document channel. A quotation response containing text engineered to influence normalisation, evaluation weighting or the content of a subsequent query is not an exotic scenario; it is the natural extension of practices that already exist in bid documents.

The controls that matter are therefore the boring ones: evaluation criteria and weights come from configuration and never from a document; extracted values are range-checked against the requirement and against other responses; anything anomalous is escalated rather than accommodated; and the evidence package presented at the award gate shows extracted values alongside the source passages, so the reviewer sees what was read as well as what was concluded.

6.4 Memory as an injection surface

Anything the agent retains across steps or runs is a persistence mechanism for an error or an injected instruction. Two rules contain this. Working memory is scoped to the run and discarded at its end, so a compromised context cannot outlive the task. Durable memory — anything written back to the graph — is governed as an assertion with provenance and is not treated as a premise merely because the system produced it.

The related discipline is that an agent's own prior output must never enter a later context as unmarked fact. It carries its status with it: this was extracted, this was inferred, this was confirmed by a human. A conclusion that loses its provenance on the way into the next step has been laundered into a premise.

7. Observability and stopping

7.1 The action log

Once most actions are never reviewed, the action log stops being a debugging aid and becomes the primary record of what the organisation did. Its content should be specified deliberately: the plan under which the step ran and the plan version, the tool and its full parameters, the evidence and authorisation relied on, the idempotency key, the outcome including postcondition verification, and any compensation subsequently applied.

The log should be written before the action is attempted, not after it succeeds. An action whose outcome is unknown is exactly the case that must be recoverable, and it is the case that a success-only log cannot represent.

7.2 Reconciliation

Reconciliation compares what the system believes it did against what the target systems record, on a schedule independent of the runs themselves. It catches the failures that no in-line check can: silent partial writes, actions rejected downstream after apparent acceptance, and effects reversed by another actor.

The reconciliation points must exist in the architecture from the start. Adding reconciliation to a system that was not designed with it means inferring intent from logs that were written for another purpose, and the inference is usually not possible.

7.3 What stop means

Every acting system needs a way to be halted, and the requirement is more subtle than a switch. At least three behaviours are distinct and all three are needed:

•    Drain. Accept no new runs; allow in-flight runs to complete. The safe default in most situations, because a completed run leaves a coherent state.

•    Halt at the next boundary. Stop each run at its next step boundary, leaving it resumable. Appropriate when something is wrong but partial states are tolerable.

•    Abort and compensate. Stop immediately and run compensation for everything already done. Appropriate when the system is known to be acting wrongly, and the most likely of the three to fail if compensation has never been exercised.

A stop capability that has not been tested under load, mid-run, is a claim. It should be exercised deliberately and periodically, in the same spirit as a protective function proof test, and the time to effect should be measured rather than assumed.

8. Build sequence

Step 1 — Specify the tool surface first

Before choosing a framework or writing a prompt, write the tool specifications: name, parameters and their types, preconditions, postconditions, reversibility class, tier, and compensating action. This document is the safety architecture. Everything else is implementation.

Step 2 — Build the policy engine before the planner

A deterministic validator with a hard-coded plan is more useful than a sophisticated planner with no validator, because the first can be extended safely and the second cannot be constrained afterwards. Build the enforcement, then build what it enforces against.

Step 3 — Idempotency and the action log at the same time as the first tool

Both are architectural and neither can be retrofitted meaningfully. The first state-changing tool should carry an idempotency key and write its intent before acting; if it does not, the pattern for every subsequent tool has been set.

Step 4 — Compensation with the tools it compensates

A tool is not complete until its compensating action exists and has been tested. Treat an untested compensation as an incomplete tool, and do not let it into a run.

Step 5 — Reconciliation before autonomy

Run the system in a mode where every action is gated, and reconcile continuously. This establishes that reconciliation works and produces a baseline of what normal looks like, both of which are needed before any action is allowed to proceed unreviewed.

Step 6 — Relax gates by tier, with limits

Move A1 actions to autonomous first, then A2 within conservative rate and cumulative limits, monitoring reconciliation discrepancies at each stage. A3 requires a standing authorisation with the four conditions of paper one. A4 never moves.

Step 7 — Exercise the stop

Before scaling, halt the system mid-run under realistic load, in each of the three modes, and measure what happens. Whatever is discovered here would otherwise have been discovered during an incident.

9. Closing observations

The architecture described here is conventional in the sense that matters: it resembles the way any other consequential automated system in an industrial facility is built. Effects are produced through a bounded interface. Constraints are enforced by components that cannot be argued with. Every action leaves a record written before the attempt. Recovery paths are defined and tested in advance rather than devised under pressure. The novelty is confined to the planner, and the planner is deliberately the part with the least authority.

That last inversion is the whole design. The most capable component in the system is given the least power to act unilaterally, not because it is untrustworthy in some general sense, but because its behaviour is not enumerable and therefore cannot be assured. Everything that must hold is placed in components whose behaviour is enumerable, and those components are unremarkable — a validator, a type system, a retry policy with keys, a registry of reverse actions, a reconciliation job.

There is a strong temptation to build in the other direction: give the agent broad tools, express the constraints as instructions, and rely on the model's judgement, because it demonstrates far better and is much faster to stand up. Systems built that way work, often for a considerable time, and their eventual failure is not a gradual degradation but a specific action that nothing was in a position to prevent.

Constrain what can be expressed. Everything else is commentary.

About this paper

This is the second paper in "Systems That Act", following "The Commitment Point". The remaining papers address multi-agent and inter-organisational operation, where accountability diffuses across systems no single party owns, and assurance and operations for systems that change state. Examples are illustrative and generic; they do not describe any specific client implementation. The paper is vendor-neutral: capabilities are described functionally and no specific platform, model, agent framework or protocol is recommended.

Read more