12-factor agents
Reliable agent software looks like reliable software: you own the prompts, the context and the control flow.
Words on this page (8)
- agent
- : A model run in a loop that chooses its own tools and steps until the task is done.
- prompt
- : The input text you give a model, including instructions, examples and data.
- context window
- : The maximum number of tokens a model can look at in one call: your prompt, the conversation, tool results and the answer together.
- tool use
- : The model asks your program to run a function and reads the result.
- tool
- : A function the model can ask your program to run, described by a name, description and argument schema.
- structured output
- : Model output that follows a schema, such as valid JSON with fixed fields.
- system prompt
- : Standing instructions placed at the start of every call that set the model's role and rules.
- token
- : A piece of text the model reads and writes, usually a word or part of a word. About 4 characters of English on average.
The ideas
The 12-factor guide, modeled on the 12-factor app, argues that good production agents are mostly regular code with model calls at specific points. Own your instead of hiding them in a framework. Own your and decide what enters it. Treat as your code executes. Keep state in your own store and make the agent resumable. Make it easy to pause for human approval. Make errors visible in context so the model can recover. Prefer small, focused agents.
Why it matters
Frameworks accelerate prototypes but can hide the prompt and the loop, which is where the bugs live. Many teams move the core loop into their own code once it needs to be reliable. Whether you use a framework or not, you should be able to print exactly what the model saw on any call.
A worked example: owning the context
Before each model call, your code assembles the messages: , retrieved documents, recent history. You can print them. If a customer reports a strange answer you reproduce the exact input and see why. This visibility is more valuable than any framework convenience.
Small agents
An agent that does one thing with three is easier to test, cheaper and more reliable than one that does everything. Chain small agents with plain code between them.
Assembling the context
Your code builds each call's messages. Invented budget for an 8,000 window:
- System prompt: 500 tokens.
- Retrieved documents: up to 3,000 tokens.
- Recent history: whatever remains, newest first.
- Tool results: trimmed to the part the model needs.
When history does not fit, your code drops the oldest turns on purpose. A framework might cut something else silently. If you decide it, you can print it.
Show errors, then cap retries
A tool fails with "file report.txt not found". Put that message in the context and the model can retry with the right name. If your code swallows it and returns "something went wrong", the model guesses.
Also cap retries, say at three, then stop and ask a person. Without a cap, a model can repeat the same failing call until the budget is gone.
Resumable state
An agent crashes at step 6 of 9. If you saved its state after each step, restart at step 7. If not, it reruns steps 1 to 6, and a step that sent an email sends it twice. Save progress after every step, and make steps that change the world safe to repeat.
When this is overkill
A single model call with no loop and no tools needs none of this. Print the prompt and move on.
Common mistake
Not being able to answer "what exactly did the model see?"
A developer builds a prototype with a heavy framework. It works until a bug appears and they cannot find what prompt the framework actually sent. They rewrite the core loop by hand in 80 lines and the bug is obvious in ten minutes. Reliable agent software tends to be plain code with a few model calls where you can see everything.
Check yourself
What should you always be able to do on any model call?
Not quite. Owning the context makes bugs findable.
When is a framework risky?
Not quite. Hidden prompts and control flow make failures harder to diagnose.
A bug appears and the framework builds the prompt internally. What is the first step?
Not quite. You need to see what the model saw before you can reason about the bug.
You got 0 of 3 right.
Go deeper
- Repo
HumanLayer · The twelve principles with examples.
- ArticleBuilding effective agentsArticle
Anthropic · Recommends starting with direct API use before frameworks.
Spotted a mistake or something unclear? Report it on GitHub.