Abstract. Most production LLM prompts are still written the way they were three years ago: someone drafts an instruction, runs it against a handful of examples, tweaks the wording when something looks off, and ships it. That process has a ceiling, and teams that hit it usually reach for the same next step, a bigger model or a fine-tuned one, both of which cost more. A 2025 multi-task study instead tested what happens when the prompt itself is optimized programmatically, against a measured objective, the way you'd optimize any other piece of a production system. On a routing-agent task, accuracy went from 85.0% to 90.0%. On an LLM-judge evaluation task, it went from 46.2% to 64.0%. Same model, same data, no fine-tuning. The newest optimizer behind that kind of result, GEPA, replaces reinforcement learning's scalar reward with natural-language reflection and, in its own six-task benchmark, beat a leading RL baseline using up to 35x fewer optimization rollouts. Dropbox ran it against a live production system in March 2026 and published the numbers. This post walks through the mechanism, the published results, and a working code path to try it against your own router or judge prompt. Research question. When a prompt is optimized programmatically against a measured objective instead of hand-tuned by inspection, how much does task accuracy actually move, how does the optimization cost compare to reinforcement-learning fine-tuning, and where does the resulting saving show up in an LLM bill? The plateau manual prompt engineering hits. Hand-tuning a prompt works well for the first few iterations and then stops working, for a structural reason rather than a skill one: a person can hold maybe a dozen examples in their head at once, and "does this new wording break any of the cases the old wording handled" is exactly the question manual iteration is worst at answering. Swap the underlying model, and the problem repeats from scratch, because wording that worked well against one model's quirks doesn't transfer cleanly to another's. A 2025 paper framed the fix as a category change rather than a technique: "Is It Time To Treat Prompts As Code?" (arXiv:2507.03620) tested DSPy, an optimization framework that treats a prompt as a compiled artifact rather than freehand text, across five real tasks: guardrail enforcement, hallucination detection in code, code generation, routing agents, and prompt evaluation. The framework's shape is simple. You declare a signature: what goes in, what comes out, and a short description of the task. You attach a metric: a function that scores an output against a labeled example, ideally with a reason attached to a bad score, not just a number. An optimizer then searches wording and few-shot example selection to maximize that metric against a held-out validation set, the same train/validation discipline any other ML system uses, applied to text instead of weights. Reflection instead of policy gradients: what GEPA actually changed. The optimizer that's driven most of the recent attention is GEPA (Genetic-Pareto, arXiv:2507.19457), accepted as an ICLR 2026 Oral. Its mechanism is a genuinely different idea from the reinforcement-learning methods it's compared against, not just a faster search. A reinforcement-learning optimizer like GRPO (the method behind DeepSeek-R1's post-training) samples many rollouts, scores each with a single scalar reward, and nudges the policy in the direction that scalar suggests, learning from a thin signal repeated many times. GEPA instead samples a full execution trace, an LLM's reasoning, tool calls, and outputs on one example, and has a reflection step read that trace in natural language and diagnose specifically what went wrong, then proposes a targeted prompt edit from that diagnosis before testing it against a Pareto frontier of prior candidates. The efficiency gap between those two approaches is the headline result. Across six tasks, GEPA outperformed GRPO by 6% on average and by up to 20%, using up to 35x fewer rollouts, and outperformed MIPROv2, the prior leading DSPy optimizer, by more than 10% on average and up to 12 points on AIME-2025. Source: arXiv:2507.19457, "GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning," accepted ICLR 2026 Oral. A single reflective trace simply carries more information than one scalar reward, and that information density is what lets GEPA get to a better prompt in a fraction of the sampled calls, which matters directly for cost: those rollouts are LLM API calls, and an optimizer that needs 35x fewer of them to reach a better result is a 35x cheaper optimization loop, before the optimized prompt has served a single production request. Two of five tasks moved cleanly, and one of them was a router. Back to the five-task study. Not every task benefited equally, and the pattern in which ones did is informative on its own. Bar chart comparing accuracy before and after DSPy optimization on two tasks: a routing-agent prompt moved from 85.0% to 90.0%, and a prompt-evaluation judge moved from 46.2% to 64.0%. | Task | Before | After | Change | |---|---:|---:|---:| | Prompt evaluation (LLM-judge criterion) | 46.2% | 64.0% | +17.8 pts | | Routing agent (model-tier selection) | 85.0% | 90.0% | +5.0 pts | | Guardrail enforcement | — | — | minor gains | | Hallucination detection in code | — | — | selective gains | | Code generation | — | — | task-dependent | Source: arXiv:2507.03620, "Is It Time To Treat Prompts As Code? A Multi-Use Case Study For Prompt Optimization Using DSPy," 2025 The two clean movers share a trait the other three lack: a crisp, checkable notion of correct. A routing agent's job reduces to one label per request, which model tier should handle this, and a judge's job reduces to one score per output, does this meet the criterion. Both are exactly the shape of objective an optimizer's metric function can score without ambiguity. Guardrail enforcement and hallucination detection are fuzzier by nature, and code generation's correctness depends on execution context an isolated metric doesn't always see, which is consistent with those three showing smaller, harder-to-generalize gains in the same study. The lesson isn't that DSPy only works on two kinds of tasks, it's that the size of the win tracks how measurable the task already is. The production test: Dropbox's relevance judge, before and after. A benchmark result is one thing; a team shipping it against a live system is another. Dropbox published exactly that in March 2026: their Dash product relies on an LLM-as-judge to score search-relevance quality, and manual prompt tuning had plateaued, with model swaps carrying real regression risk because a prompt tuned by eye against one model's behavior didn't reliably transfer to the next one. They optimized the judge prompt against a measured objective, minimizing disagreement with human annotations while keeping the output's JSON well-formed, using DSPy. Against gpt-oss-120b, NMSE (a lower-is-better disagreement metric) dropped from 8.83 to 4.86, a 45% reduction. Against the smaller gemma-3-12b, the gap was larger: NMSE fell from 46.88 to 17.26, and the model's malformed-JSON rate, output that failed to parse and had to be retried, dropped from 40% to under 3%. Bar chart showing gemma-3-12b's malformed-JSON output rate as Dropbox's relevance judge: 40% with a hand-written prompt, under 3% after DSPy-based optimization. That's not a benchmark number, it's a 97% relative drop in retries on a model cheap enough that Dropbox wanted to run it in the first place, and a prompt that got there without switching models or collecting new training data. The other number worth sitting with is speed: adapting the judge to a new model dropped from one to two weeks of manual iteration to one to two days, and the team reported enough headroom in the labeling budget to score 10 to 100 times more evaluation data at the same cost. Source: Dropbox Tech, "How we optimized Dash's relevance judge with DSPy," March 17, 2026. Where this actually shows up in a token bill. None of the numbers above are denominated in dollars, and they don't need to be to explain where the savings land. The optimization loop itself is cheaper. GEPA's rollout efficiency, up to 35x fewer sampled calls than GRPO for a comparable or better result, means the search for a good prompt costs a fraction of what a reinforcement-learning pass over the same task would cost, and that search is pure LLM API spend with no production value until it finishes. A more accurate router misroutes less. Blind or poorly calibrated routing is exactly what turns a cost-saving lever into a quality risk: send a request to too cheap a model and a wrong or incomplete answer forces a retry on a stronger one, which bills twice. Moving a routing agent's accuracy from 85.0% to 90.0% is a direct cut in how often that expensive failure mode fires, without touching which models are in the pool. Fewer malformed outputs means fewer retries. Structured-output failures aren't free; a response that fails to parse is a wasted call, plus the retry that has to redo it. A 97% relative drop in malformed JSON, as Dropbox measured on their smaller model, is a 97% relative drop in that specific waste category. Faster model migration lowers the cost of switching to something cheaper. A prompt that takes one to two days to re-validate against a new model, instead of one to two weeks, is far less friction standing between a team and adopting whatever this quarter's cheapest capable model turns out to be, which is the whole premise behind routing to it in the first place. Manual tuning vs. programmatic optimization, side by side. | | Manual prompt engineering | DSPy / GEPA | |---|---|---| | How it iterates | Edit text, eyeball a handful of examples | Optimizer searches against a scored validation set | | What "better" means | A person's judgment call | A metric function, checkable and repeatable | | Cost to adapt to a new model | 1-2 weeks, reported by Dropbox | 1-2 days, reported by Dropbox | | Optimization compute | None, but no measurement either | Rollouts against the target model; GEPA needs up to 35x fewer than RL fine-tuning | | What changes | Instruction wording, by feel | Instructions and few-shot examples, jointly searched | | Requires new training data | No | A labeled example set with a scoring function | | Portable to a different model | Not reliably | Recompile against the new target | Try it: optimizing a routing prompt with dspy.GEPA. The five-task study's routing-agent result is directly reproducible against your own model-selection logic. A minimal version: import dspy lm = dspy.LM("openai/gpt-4o-mini") dspy.configure(lm=lm) class RouteRequest(dspy.Signature): """Pick the cheapest model tier that can still handle this request correctly.""" request: str = dspy.InputField() tier: str = dspy.OutputField(desc="one of: small, medium, large") router = dspy.Predict(RouteRequest) def metric_with_feedback(gold, pred, trace=None, pred_name=None): correct = pred.tier == gold.tier feedback = ( "Correct tier." if correct else f"Picked {pred.tier}, should have been {gold.tier}. " f"Reconsider what in the request signals its real complexity." ) return dspy.Prediction(score=float(correct), feedback=feedback) optimizer = dspy.GEPA( metric=metric_with_feedback, max_metric_calls=250, reflection_minibatch_size=3, reflection_lm=dspy.LM("openai/gpt-4o-mini"), track_stats=True, ) train_examples / val_examples: list[dspy.Example] with .request and .tier set optimized_router = optimizer.compile( router, trainset=train_examples, valset=val_examples, ) The trainset and valset are the same kind of labeled data any routing evaluation needs: a request paired with the tier a human or a stronger model would have picked for it. What GEPA adds over a static prompt is the reflection loop, every misrouted example in metric_with_feedback returns a specific, readable reason, and the optimizer uses that reason, not just the 0/1 score, to propose the next edit. Nadir runs a version of this same problem, deciding which model tier a request needs, on every call by default, and reports the cost delta against an always-premium baseline per request. A team tightening its own router or judge prompt with GEPA doesn't need to also build that decision layer from scratch to see where the resulting accuracy gain turns into dollars. Conclusion. The gap this research points at isn't between good and bad prompts, it's between prompts that were measured and prompts that weren't. A hand-tuned instruction can be excellent, but there's no way to know how excellent without the same discipline any other production component gets: a metric, a held-out set, and a repeatable way to check whether a change helped. Where that discipline got applied, in a routing agent and a judge, the gains were large and the mechanism, GEPA's reflection-driven search, needed a fraction of the optimization compute reinforcement learning does to get there. Where a task's notion of correct stayed fuzzy, the gains stayed modest, which is itself useful information; it tells you where hand-tuning still has to carry the weight. Dropbox's production numbers, a 97% relative drop in one model's malformed-output rate and a model-adaptation cycle cut from weeks to days, are what that discipline looks like once it leaves the benchmark and starts shipping.