Why ad-hoc prompting fails
The standard approach to giving an AI agent work: write a prompt describing what you want. Maybe add some context. Hit send. Hope the agent figures out the rest.
This works for small tasks. Write a function. Fix a bug. Run a test. The scope is narrow, the success criteria are obvious, and the agent can verify its own work.
It does not work for anything larger. A multi-file refactoring across three services. A new feature that requires reading existing code, understanding constraints, writing implementation, and validating against acceptance criteria. A deployment that involves infrastructure manifests, image builds, and smoke tests.
For these tasks, ad-hoc prompting fails in predictable ways:
Scope creep. Without explicit boundaries, the agent interprets “fix the routing logic” as “fix the routing logic and also refactor the error handling and also update the tests and also clean up the imports.” Each addition is individually reasonable. Together they produce a PR that nobody asked for and nobody can review.
Missing context. The agent does not know what it does not know. Without a structured context-loading step, it guesses at constraints, invents conventions, and makes assumptions about the codebase that are wrong. The output looks correct until you read it against the actual architecture.
Ambiguous completion. When is the task done? The agent thinks it is done when the code compiles. You think it is done when the tests pass, the deployment succeeds, and the smoke test returns green. Without explicit acceptance criteria, “done” is a negotiation that happens after the work is complete.
No escalation path. The agent encounters something it cannot resolve. A dependency is missing. A service is down. A decision requires human judgment. Without an explicit escalation model, the agent either blocks silently (wasting the session) or makes a guess (introducing risk).
Ad-hoc prompting treats the agent as a conversation partner. ISAs treat the agent as a contractor with a scope of work, a set of constraints, and a definition of done.
Anatomy of an ISA
ISA stands for Intelligent Service Agreement. The name is deliberate. It borrows from the service agreement pattern in IT service management (ITIL) and from the legal structure of a contract: scope, constraints, acceptance criteria, escalation triggers, completion definition. The “intelligent” qualifier means the agreement is written for an agent that can reason about its own compliance with the terms.
graph TB
ISA[ISA Document]
ISA --> I[Intent<br/>Why this exists]
ISA --> O[Outcome<br/>What done looks like]
ISA --> AB[Autonomy Boundary<br/>Always / Ask / Never]
ISA --> AC[Acceptance Criteria<br/>ATDD format]
ISA --> EX[Execution Loop<br/>Phased tool sequence]
ISA --> CG[Completion Gate<br/>Checklist before close]
style ISA fill:#1a1917,stroke:#00c8a0,color:#e8e5de
style AB fill:#1a1917,stroke:#c9a84c,color:#e8e5de
Intent
One paragraph. Why does this work item exist? Not what needs to be built. Why it needs to be built. The agent reads this first and uses it to resolve ambiguity throughout execution. If a decision could go either way, the intent section breaks the tie.
Outcome (What Done Looks Like)
Falsifiable statements. Not “the service works.” Statements like: “the API returns a 200 status with a valid response body for each of 3 test inputs.” The agent can verify each statement independently. If any statement is false after execution, the work is not done.
Autonomy boundary
Three tiers, explicitly enumerated:
Always: actions the agent takes without checking. Read the codebase. Search project memory. Write implementation files. Run tests. Commit and push. These are within the agent’s authority.
Ask First: actions that require explicit human approval before execution. Changing behavior rules. Modifying thresholds. Adding new data categories. Wiring capabilities not in scope. These are design decisions, not implementation decisions.
Never: hard stops. Actions the agent must not take under any circumstances. Storing private data with public classification. Using a more expensive model tier than specified. Violating the memory wall between contexts. These are not negotiable regardless of what the agent thinks is “better.”
Acceptance criteria (ATDD format)
Each criterion follows a Given-When-Then structure borrowed from Acceptance Test-Driven Development. The format forces specificity:
AC-1: Agent responds to messages in the correct channel
Given: the messaging service is deployed and running
When: a user sends a message to the designated channel
Then: the agent routes the message to the correct handler
And: the response matches the expected tone and format
And: the service logs confirm the route was triggered
Negative: the agent must NOT respond to messages in other channels
Measure: 3 test messages sent, 3 responses received, 3 log entries
The Negative line is as important as the positive criteria. It defines what the system must not do, not just what it must do. The Measure line defines how verification works.
Execution loop
A phased tool sequence that tells the agent exactly what to do and in what order. Phases are sequential. Within a phase, steps can be parallel or sequential, explicitly marked. Error handling is defined per phase with recovery actions and escalation triggers.
This is not a suggestion. It is a plan. The agent follows the plan. Deviations from the plan require Ask First escalation.
Completion gate
A checklist of all acceptance criteria plus operational requirements (code committed, tests passing, deployment verified, work item closed in the tracking system). The agent does not declare “done” until every line in the completion gate is checked.
How ISAs compose with work tracking
Every work item, from a one-hour fix to a multi-session build, gets a tracking ID. The tracker holds status, priority, dependencies, and ownership. ISAs attach to tracked work items.
sequenceDiagram
participant Human
participant Tracker as Work Tracker
participant ISA as ISA Document
participant Agent as Agent Fleet
Human->>Tracker: Create work item
Human->>ISA: Write ISA for work item
Human->>Agent: Dispatch agent against work item
Agent->>ISA: Read ISA (Phase 0)
Agent->>Agent: Execute phases 1-N
Agent->>Tracker: Close work item on completion
Agent->>Human: PR opened / results posted
The tracker contains metadata: what the work is, when it was created, what it depends on, what its priority is. The ISA contains the execution contract: how the work gets done, what the constraints are, what done means.
Separating the two is deliberate. The work item can exist without an ISA (for tracking purposes). The ISA can exist without immediate dispatch (for planning purposes). When the agent is dispatched, it reads both: the work item for context, the ISA for instructions.
How an agent fleet consumes ISAs
An agent fleet is a set of AI coding instances dispatched against work items. Each instance gets a task description and a behavioral template that defines how it operates. The templates tell the agent how to behave. The ISA tells the agent what to build.
Phase 0: Memory check. Before touching any code, the agent searches project memory for prior work on this item. This catches the case where a previous agent session started the work, hit a failure, and left breadcrumbs. Without Phase 0, each dispatch starts from scratch. With Phase 0, each dispatch has the context of every previous attempt.
Phase 1: Read. The agent reads the relevant codebase. Not guesses about the codebase. Actually reads the files, in parallel, and waits for all reads to complete before proceeding. The ISA specifies which files to read. This eliminates the “agent invents the architecture” failure mode.
Phase 2: Check dependencies. Verify that prerequisite work items are closed, prerequisite PRs are merged, prerequisite services are running. If any check fails, stop and surface to the human. Do not proceed on assumptions.
Phases 3-N: Execute. Write code, run tests, deploy, verify. Each phase has explicit completion criteria. Error handling blocks define recovery for each anticipated failure mode.
Final phase: Close. Save a memory milestone (so future dispatches have context). Close the work item in the tracking system. Post results. The session ends with a clean state.
The ISA execution loop is not a guide. It is a contract. The agent follows the phases. Deviations require explicit escalation. This is the governance that ad-hoc prompting lacks.
Batch validation: 11 ISAs in one session
The proof that the ISA pattern scales came from writing 11 ISAs in a single working session, each covering a distinct build with its own acceptance criteria.
The ISAs spanned persona routing, session lifecycle management, notification pipelines, agent template upgrades, memory persistence layers, watchdog systems, event buses, and semantic analysis. Each one followed the same structure: intent, outcome, autonomy boundary, ATDD acceptance criteria, phased execution loop, completion gate.
The consistency of the format meant that any agent instance could pick up any ISA and know exactly how to execute it. The dispatcher did not need to understand the content of each ISA. It only needed to know the dependency graph and the dispatch order.
Lessons on ISA granularity
Not all 11 ISAs were the same size. Some (a template syntax fix) were small: one repo, one change, well-defined before and after. Others (a session lifecycle layer) were large: three repos, five sub-items, complex dependencies.
Lesson 1: One ISA per deployable unit. The right granularity is: one ISA produces one PR (or one set of coordinated PRs) that can be deployed and verified independently. When the session lifecycle ISA was written as a single ISA with five sub-items, it was too large. Each sub-item needed its own ISA because each was independently deployable. The parent ISA became a coordination document, not an execution contract.
Lesson 2: Acceptance criteria are the granularity test. If you cannot write ATDD acceptance criteria that are independently verifiable, the ISA is either too large (split it) or too vague (sharpen it). The Given-When-Then format forces falsifiability. If the criterion is not falsifiable, it is not a criterion.
Lesson 3: The autonomy boundary is the hardest section to write. Deciding what the agent can do without asking is a governance decision with real consequences. Too permissive and the agent makes decisions you did not authorize. Too restrictive and the agent blocks on trivial questions, burning session time. The right calibration comes from experience. After 11 ISAs, the pattern stabilized around: “implementation decisions are Always; design decisions are Ask First; data classification decisions are Never.”
Lesson 4: Phase 0 (memory check) is non-negotiable. Multiple dispatches against the same work item is the normal case, not the exception. Agent sessions time out, hit errors, run into unmet dependencies, and get restarted. Without Phase 0, each restart reinvents context. With Phase 0, each restart reads the breadcrumbs from the last attempt and picks up where it left off. Persistent memory breadcrumbs are the session continuity mechanism for autonomous agents.
Structured error handling in ISAs
Each ISA includes an error handling block per phase. The block defines: the error, the context (which phase), the recovery action, and whether to escalate to the human.
flowchart TD
E[Error Detected] --> C{Recovery defined?}
C -->|Yes| R[Execute Recovery]
R --> V{Recovered?}
V -->|Yes| Continue[Continue Phase]
V -->|No| Esc[Escalate to Human]
C -->|No| Esc
Esc --> Wait[Wait for Human Decision]
style E fill:#1a1917,stroke:#c1121f,color:#e8e5de
style Esc fill:#1a1917,stroke:#c9a84c,color:#e8e5de
style Continue fill:#1a1917,stroke:#00c8a0,color:#e8e5de
Example from a persona implementation ISA:
Error: prerequisite PR not merged
Context: Phase 2 dependency check
Recovery: Stop. Surface to human: "Prerequisite PR is not
merged. This feature depends on it. Merge first."
Escalate: Yes
Error: system prompt exceeds token budget
Context: Phase 5 token audit
Recovery: Trim the base prompt. Preserve voice rules.
Cut examples and elaborations first. Rerun count.
Escalate: No
Error: upstream service unreachable (state fetch fails)
Context: Phase 6 smoke test
Recovery: Use neutral defaults. Log warning. Continue.
Escalate: No
The escalation flag is the key design decision. Some errors are recoverable by the agent (trim the prompt, use defaults). Some errors require a human decision (unmet dependency, service down, scope question). The ISA makes this distinction explicit rather than leaving it to the agent’s judgment.
The ISA does not assume the agent will succeed. It assumes the agent will encounter failures and provides a decision tree for each one. That is the difference between a prompt and a contract.
ISA as a governance pattern
ISAs are not just a prompting technique. They are a governance pattern for autonomous AI agents.
Auditability. Every ISA is a document that can be reviewed before execution. The autonomy boundary is explicit. The acceptance criteria are falsifiable. The error handling is defined. If the agent produces an unexpected outcome, the ISA provides the reference against which to evaluate what went wrong.
Reproducibility. Given the same ISA and the same codebase state, two different agent instances should produce equivalent output. The ISA constrains the agent’s decision space enough that implementation variance is bounded. This is not deterministic (LLMs are stochastic), but it is reproducible enough for practical purposes.
Composability. ISAs compose through the dependency system. Complex work decomposes into multiple ISAs with declared dependencies. The dispatcher can schedule ISAs in dependency order without understanding the content. This is the mechanism that turns one person’s architecture decisions into a fleet’s work queue.
Portability. ISAs are markdown documents. They are not tied to a specific agent framework, a specific LLM, or a specific deployment platform. They could be consumed by Claude Code, by Cursor, by a custom agent, by a human engineer. The format is the interface. The execution engine is pluggable.
ISA template for practitioners
If you are building autonomous agents and want to try the pattern, here is the minimal viable ISA structure:
# ISA: [work-item-id] -- [title]
## Intent
Why this work exists. One paragraph.
## Outcome (What Done Looks Like)
Falsifiable statements. Numbered list.
## Autonomy Boundary
### Always (agent executes without asking)
### Ask First (requires human approval)
### Never (hard stop)
## Acceptance Criteria
### AC-1: [name]
Given: [precondition]
When: [action]
Then: [expected result]
Negative: [what must NOT happen]
Measure: [how to verify]
## Execution Loop
Phase 0: Memory check (search for prior work on this item)
Phase 1: Read current state (specified files, not guesses)
Phase 2: Check dependencies (prerequisite items closed?)
Phase 3-N: Execute (write, test, deploy, verify)
Final: Close (save memory milestone, close work item, post results)
## Error Handling
Per phase: error, context, recovery action, escalate yes/no
## Completion Gate
- [ ] AC-1 verified
- [ ] AC-2 verified
- [ ] Code committed
- [ ] Tests passing
- [ ] Work item closed
Adapt as needed. The sections are the point, not the syntax. The autonomy boundary and acceptance criteria are the minimum viable governance layer. Everything else is optimization.
Open questions
ISA generation. Currently ISAs are written by hand. This is fine for a batch of 11. It does not scale to 50. Can the ISA format itself be generated from a higher-level intent description? The risk is that generated ISAs lose the specificity that makes them useful. The opportunity is that the pattern is structured enough to be partially automatable.
ISA versioning. What happens when an ISA needs to change mid-execution? Currently the answer is: write a new ISA. But if the agent is mid-session on the original ISA, there is no mechanism for hot-updating the contract. The agent finishes the old ISA or discards it. There is no graceful transition.
Cross-agent ISAs. What happens when multiple agents with different capabilities need to coordinate on a single deliverable? The dependency model handles sequential handoffs. It does not handle parallel collaboration where two agents work on the same codebase simultaneously. Coordination protocols for multi-agent ISA execution are an open design problem.