Harness Engineering

You might say

The same AI works well sometimes but repeats the same mistake elsewhere. What can I change besides the model?

The engineering practice of turning project knowledge, tools, state, permissions, and checks into a working environment an AI agent can useFor example, when a frontend agent delivers a plausible page, an independent evaluator can operate the live flow and return failed steps and screenshots to the next turn. Harness engineering changes the conditions around the model rather than making the prompt longer.

Know first

AI Agent Agent Loop Tool Calling

Agent Harness EngineeringAgent Harness

What a harness adds when an agent keeps failing

CLASSIC FAILUREThe page looks finished, but the main flow is brokenAnthropic · frontend application harness

WHAT YOU CAN SEE

A static screenshot looks plausible
Clicking Play does nothing
The agent still reports “done”

playwright.click("Play")
✗ no state change
functionality: 1 / 5

ADD TO THE HARNESS

1Independent evaluator2Playwright interaction3Screenshots + four criteria

FEEDBACK TO THE NEXT TURNThe evaluator returns the failed click, screenshot, and critique to the generator for another pass.

WHAT COUNTS AS BETTERA person can complete the main flow

The run ends only after the live page works and the evaluator’s check passes.

A screenshot proves that a page can render. An evaluator that actually operates it proves whether the experience works.

When a failure repeats, improve the environment that exposes, records, and corrects it—not just the prompt.

What each classic case fixes

The frontend renders but does not work: An independent evaluator operates the live page, captures screenshots, and checks the complete flow. Failed actions and specific critique return to the next turn instead of letting the generator grade itself.

A long task forgets between sessions: A feature list, progress file, startup script, and Git commits keep state in the project. The next session reads them, runs a basic check, and continues one unfinished item.

A large repository drifts: A short project map provides the entry point, structured docs hold the details, and custom lint and structural tests turn architecture rules into failures with a remediation path.

Quick checkChoose the best answer

A frontend agent delivers a page that “looks done” three times, but the main button never works. Which change is harness engineering?

Add “carefully check every interaction” while still letting the agent judge its own workHave an independent evaluator complete the live flow, return failed steps and screenshots, and end only after it passesOnly ask for more static screenshots and pass the page whenever they look complete

You can say this to an AI Agent

Do not rebuild the feature yet. Describe the repeated failure as: observable evidence → missing environmental support → file, tool, or check to add → how the result returns to the next turn → what counts as passing. Propose the smallest harness first, then implement it and verify it with real evidence.

Learn next

1Agent Loop→2Human-in-the-loop→3CI→4End-to-End Test→

Want GPT or Codex?VibeHubGPT support

GPT-6 Astra

Build in 3D. Put your computer to work.