Stop Vibe-Checking Your Agents: Eval-Driven Prompt Optimization with DSPy

Most teams still improve agent prompts the same way: change some wording, run it a few times, eyeball the outputs, and ship it if it "feels better." That is agent prompt evaluation by vibes — and it breaks the moment a change silently regresses, or a new model underneath your agent quietly shifts its behavior. There is a more disciplined path: treat prompts like code, with an eval set, real metrics, and regression tests, and let a tool like DSPy optimize them against that signal instead of hand-tuning.

In July 2026, Simon Willison published a concrete walkthrough of exactly this — using DSPy to evaluate and improve the SQL prompts in his Datasette Agent. This post generalizes that example into a repeatable, eval-driven workflow you can apply to any agent, whether it writes SQL, browses the web, or ships code.

Why hand-tuning agent prompts fails at scale

Hand-tuning feels productive because you always find a prompt that looks better on the two or three examples you happened to try. The problems show up later:

You have no objective signal. "Feels better" doesn't tell you whether accuracy went up, or whether you just got lucky on the examples you looked at.
Changes regress silently. A tweak that helps one case can quietly break three others you didn't re-check.
The ground shifts under you. Swap the underlying model — or the model gets updated — and a prompt hand-fitted to the old one can degrade with no warning.

Willison's own commentary from July 2026, "Better Models: Worse Tools," captures the tension: a stronger base model does not automatically mean your tool or agent gets better, because behavior you had implicitly tuned around can change. Without an eval harness, you find out in production.

What is eval-driven agent development?

Eval-driven agent development means you define what good looks like before you optimize, and you measure every change against it. Concretely, it has three parts:

An eval set — a collection of representative tasks with a way to judge success.
A metric — a function that scores the agent's output on each task (pass/fail, task success rate, cost, latency).
A loop — you change something (the prompt, the model, the tools), re-run the eval set, and compare scores.

Once those exist, "did this help?" becomes a number instead of an argument. And crucially, the same harness doubles as a regression test: rerun it after any change and you immediately see what got worse, not just what got better.

What is DSPy and where does it fit?

DSPy is a framework for programming with language models where you declare what you want a module to do and let the framework optimize how the prompt achieves it against a metric you define. In the eval-driven picture above, DSPy is the optimizer: given your eval set and metric, it searches for prompt configurations that score better, rather than you hand-writing and hand-testing each variation.

It is not magic, and it does not remove the need for a good eval set — if anything, it makes the eval set more important, because DSPy optimizes toward whatever you tell it to measure. Garbage metric in, garbage optimization out. Willison's Datasette write-up is a good grounding example: he used DSPy specifically to evaluate and improve the agent's SQL prompts, i.e. to push a measurable task (does the generated SQL do the right thing?) rather than to chase a subjective feel.

Building an eval set for your agent

The eval set is where most of the real work — and most of the payoff — lives.

Choosing representative tasks

Pick tasks that reflect what your agent actually does in production, including the hard and adversarial cases, not just the happy path. For a SQL agent that means a spread of real question-to-query tasks across schemas and difficulty. For a coding agent it means real change requests. Aim for coverage of the failure modes you care about; a handful of well-chosen, judgeable tasks beats a huge set you can't score reliably.

Metrics that actually matter

Your metric has to be something you can compute automatically and that correlates with real success:

Task success / correctness — did the output actually accomplish the task (e.g. the SQL returns the right rows, the code passes tests)? This is the metric that matters most.
Pass/fail on hard constraints — did it stay within required formats, permissions, or safety rules?
Cost and latency — tokens spent and time taken. A prompt that's slightly more accurate but three times more expensive may not be a win.

Define these before you optimize. If you can't articulate how you'd score an output, you can't yet run an eval-driven loop on it.

Optimizing prompts with DSPy, step by step

Following the shape of Willison's Datasette walkthrough, the workflow generalizes to:

Express the agent step as a DSPy module — declare its inputs and the output you want, rather than hand-writing the full prompt.
Assemble your eval set of representative tasks with a scoring metric (for a SQL agent, whether the generated query produces the correct result).
Run a baseline — measure the current prompt against the eval set so you know your starting score.
Let DSPy optimize against the metric, searching prompt configurations that improve the score.
Compare optimized vs. baseline on the eval set, and inspect where it improved and where it regressed.
Keep the eval set as a regression test — rerun it on every future change.

Describe-then-optimize is the mental shift: you stop writing the perfect prompt by hand and start specifying the goal plus the yardstick, then let the optimizer do the search.

What to evaluate when a new model drops

New models arrive constantly — OpenAI previewed GPT-5.6 "Sol" in June 2026, and coding-agent runtimes iterate fast (OpenClaw's 2026.7.1 beta shipped day-one GPT-5.6 support; Claude Code pushes frequent releases). Each of those is a reason to re-run your evals, not to blindly upgrade. An eval set turns "should we adopt this model?" into an experiment: point your harness at the new model, compare scores against your current baseline, and see whether task success actually improves for your workload — and at what cost and latency.

The example eval targets are exactly these kinds of moving parts: the model your agent calls, and the runtime it runs in. When any of them changes, your eval set is what tells you whether the change was an upgrade or a regression.

Better models, worse tools? Avoiding the tuning trap

Willison's "Better Models: Worse Tools" is the cautionary half of this story. Hand-tuning a prompt to one model's quirks quietly over-fits to that model; when the model changes, the tuning can backfire. Eval-driven development is the guardrail: because you optimize against a metric rather than a single model's behavior, and because you keep the eval set as a regression test, a model swap becomes a measured comparison instead of a silent surprise. You still get the benefit of a better base model without inheriting the risk of prompts secretly welded to the old one.

FAQ

How do you evaluate an LLM agent?

Build an eval set of representative tasks with an automatic scoring metric (task success, correctness, cost, latency), run the agent against it to get a baseline score, then compare that score after every change. The key is that "better" is a measured number, not a feeling.

What metrics should I use for agent prompts?

Prioritize task success / correctness — did the output actually do the job? Add hard-constraint pass/fail checks (format, permissions, safety), plus cost and latency so you don't optimize accuracy at unacceptable expense. Whatever you choose, it must be computable automatically so you can run it repeatedly.

Do I need to re-evaluate prompts when the model changes?

Yes. A prompt tuned for one model can regress on another, and models get updated frequently. Re-running your eval set against the new model turns the upgrade decision into a measured experiment rather than a gamble.

Is prompt optimization better than manual prompt engineering?

For anything you run repeatedly, optimization against a real eval set beats manual tuning because it's objective and repeatable — a tool like DSPy searches configurations against your metric. But it depends entirely on a good eval set and metric; optimization only pushes toward what you measure. Manual insight still matters for designing the tasks and metrics in the first place.

The durable lesson here isn't DSPy specifically — it's the eval-driven workflow that outlives any single model release. Define what good looks like, measure every change against it, and keep the eval set as a regression net for the next model that drops.

If you're building agents and want to move from vibe-checks to measured improvement, Clawvard is where we go deep on agent evaluation, model comparison, and the tooling around them. Explore Clawvard to apply eval-driven development to your own agents, and follow our updates as the eval tooling landscape keeps moving.