01What Google Harness² Is and How It Works
Google Harness² is a free, open-source tool from Google Research that makes AI agents perform better without upgrading the AI model. It improves the "harness" — the instructions, rules, memory, and tools wrapped around the model.
The tool works by running the same task multiple times with different setups. It compares the results, keeps what works, and builds a final setup. This happens through a three-step loop called Propose, Probe, Compose.
The agent suggests changes, tests them, and combines the best ones. No new training or bigger model is needed.
02Main Features of Harness²
Harness² has three big ideas at its core. The first is the Propose, Probe, Compose loop. It suggests changes (Propose), tests each change on the same task (Probe), and builds a final setup from what works (Compose).
This happens without an answer key — the system learns by comparing results.
The second idea is that Harness² can edit eight parts of the agent's setup. These include the system prompt, guard rails, memory, tool guidance, sub-agents, skills, scripts, and plugins. Each part controls something the agent knows, does, or hands off.
- System prompt: main instructions
- Guard rails: rules
- Memory: what the agent remembers
- Tool guidance: how to use tools
- Sub-agents: helper agents
- Skills and scripts: saved programs
- Plugins: add-ons for each step
The third idea is the hidden second loop called harnessing-level recursion. After each task, the experience reflector saves lessons into an improvement harness. Next time, the system reads those lessons before suggesting changes.
This makes every task improve the next one.
03How to Set Up Google Harness²
- Make sure you have a Linux system with Python 3.12 installed. Node.js and Google Cloud setup may also be needed.
- Download the Harness² code from its open-source GitHub repo. It's version 0.1.0, so expect rough edges.
- Install the required dependencies using the package manager (like pip or npm). Follow the README instructions in the repo.
- Set up an executor agent. Currently Harness² supports Open Code and Codex. Choose one and configure it.
- Start with a small set of tasks to test the setup. This keeps API costs low during the probe phase.
- Run Harness² on your tasks. It will propose, probe, and compose improvements automatically.
- Review the final setup and apply it to your agent. The tool saves what worked best.
04How to Use Harness² with Examples
In the episode, Julian walks through how Harness² handles real office tasks. Imagine you have an SEO agent that writes blog posts. It misses an important point.
The fix is usually a better skill file — a set of rules saved as a program.
Harness² finds on its own: it proposes adding a rule like “always include the keyword in the first paragraph.” It tests that rule against the current setup. If the test shows better results, it keeps the change. If not, it tries something else.
- To use Harness² on a task, give it the agent's current setup and the task goal.
- Set a budget for how many probes to run. Parallel mode tests many setups at once. Sequential mode refines one setup repeatedly.
- After the run, check the improvement harness notebook. It holds lessons backed by actual results.
- Apply the new setup to your agent. Over time, every task makes the agent smarter.
05Test Results and Comparisons from the Episode
Google tested Harness² on three sets of work tasks: Work Buddy Bench (office, web, code jobs), Job Bench (professional office tasks), and Lab. They used two AI models: GLM5 and Gemini 1.5 Flash.
Results showed big gains after just one round of improvement:
- Work Buddy Bench: scores up by as much as 11.5 points
- Job Bench: scores up by 4.5 points
- Lab: partial pass rate up by 11.8 percentage points
The model weights (what the AI learned during training) never changed. No one paid to build a new model. The gains came entirely from improving the setup.
Julian notes that Google says more rounds can give even bigger gains. The project quickly gained over 12,600 GitHub stars and 1,200 forks.
06Tips, Limits, and Mistakes to Avoid
Harness² costs more to run because every probe is a full agent run. Testing six setups means six runs instead of one. Google advises starting with a small set of tasks to keep API fees low.
The tool is a research project, not a supported Google product. Version 0.1.0 is early. Expect bugs and missing features.
It's built for technical users — you need Linux, Python, and some command-line knowledge. It's not a no-code app.
A common mistake is not using the second loop. The experience reflector saves lessons, but only if you let it. Make sure the improvement harness is active so each task learns from the past.
Another mistake is letting the memory fill with unproven guesses. Harness² only keeps evidence-supported lessons, so trust those results.
07Who Should Use Google Harness² and Why It Matters
Harness² is for anyone building AI agents and wanting them to get better over time. Developers, researchers, and tech-savvy business owners can use it to improve agents without waiting for new models.
The main lesson applies even if you don't code: the setup matters as much as the model. A better kitchen (instructions, memory, skills) made the same chef up to 11.8 points better on office work. You can apply this today by refining your agent's plain-English instructions and rules, even without Harness² itself.
Julian says understanding this principle compounds over time. Each improvement makes the next one easier. For non-technical users, joining communities that build on these ideas can help.
In the AI Profit Board, members share agents that share memory and skills — the same parts Harness² improves. But the core takeaway is that you don't need to wait for a bigger brain. A smarter setup works.
