From prompts to agents to harnesses

The sequenceCopy Anchor Link

Prompt engineering (2020 to 2023): GPT-3 showed that wording and examples change results, so people tuned prompts. Retrieval and tools (2023): models gained access to documents and functions. Agents (2024 to 2025): models run in a loop, choose tools and act over many steps. Context engineering (2025): the realization that what is in the window matters more than phrasing. Harness engineering (2025 to 2026): the recognition that the system around the model, not only the model, decides how well an agent does.

What the pattern isCopy Anchor Link

Each stage moved attention outward. First the sentence you type, then everything the model sees, then the loop that runs it, then the whole environment: tools, rules, checks, sandboxes and memory. The model did not get replaced at each step. The unit of engineering got bigger.

What is real and what is hypeCopy Anchor Link

Real: the shift from judging one response to judging a whole run of many steps. Real: that most agent failures come from missing tools, vague rules or noisy context rather than a weak model. Rebranding: calling normal software engineering a new discipline. A harness has a retry loop, tests, permissions and logging, and those are old ideas applied to a new component. The honest framing is that they are the same ideas applied to a component that is non-deterministic, which makes them more important.

How to use thisCopy Anchor Link

When you meet a new buzzword, ask three things. What mechanism does it name? What can I measure to know if it helps? What would I have called this last year? The next lessons apply that test to prompt engineering, harness engineering and the buzzwords around them. The Agents chapter covers agents in depth.

A worked example: the same task, five erasCopy Anchor Link

Task: answer customer questions. Prompt era: write a clever instruction. Retrieval era: add the right help articles to the prompt. Agent era: let the model look up orders and issue refunds with tools. Context era: carefully choose what is in the window at each step. Harness era: add tests, approvals, tracing and a separate checker so the whole system is dependable. Each era added a layer. None removed the model.

A habit worth keepingCopy Anchor Link

When you meet a new term, find the mechanism, find the measurement, and find the old name. If you cannot find all three, wait.

Running the three questions on a new termCopy Anchor Link

Take an invented term, "intent engineering", from a vendor post. Apply the checks in order.

1. Mechanism: read for what the system does differently. Suppose the post says the app rewrites a vague request into a clear goal before the model sees it. That is a concrete step.
2. Measurement: ask for a before and after. Suppose the vendor reports 62 of 100 test requests solved before, and 71 of 100 after. That is 9 points better, but 100 requests is a small test. Random luck alone can move a score by several points, so ask for more runs.
3. Old name: rewriting a request before processing it is called query rewriting, and search engines have done it for years. Verdict: a real technique with a small, uncertain gain, applied in a new place. It is not a new discipline.

When the mechanism is real but the claim is notCopy Anchor Link

A pitch says: "Harness engineering lets any model act like a top engineer." The mechanism behind it is real. Tests catch wrong answers, and permissions stop bad actions. The claim goes too far, because a harness checks and limits a model. It cannot add knowledge or reasoning the model lacks.

You can test the claim. Keep the harness fixed and swap in a weaker model, then run the same twenty tasks and count passes. If the count drops a lot, the model still matters. Expect that result. A good harness raises the floor, so fewer runs fail badly, but it does not remove the gap between models.

Common mistakeCopy Anchor Link

Thinking you must learn the newest name to be current. The mechanisms underneath last longer than the labels.