Title: VeRO: A Framework for Optimizing Coding Agents

Abstract:

Agent harness optimization—an iterative process of refining a target agent by modifying and assessing its code—is emerging as a critical application for coding agents. However, the research community currently lacks a comprehensive understanding of how coding agents perform on this specific task. Unlike traditional software engineering, agent harness optimization involves a unique complexity: it combines deterministic code with stochastic large language model (LLM) completions. This hybrid nature necessitates the structured recording of both intermediate execution traces and final downstream results.

To tackle these challenges, we present VeRO (Versioning, Rewards, and Observations), an outer harness designed to capture versioned snapshots, manage evaluation budgets, and record structured execution traces for target harnesses. Additionally, we introduce VeRO-Bench, a benchmark suite comprising target agents and tasks alongside standardized evaluation procedures. Leveraging VeRO, we performed an empirical analysis to compare various optimizers across different tasks, identifying which specific modifications consistently enhance the performance of target agent harnesses. We are releasing VeRO to facilitate research into agent optimization, viewing it as a fundamental capability for coding agents. The codebase is accessible at https://github.com/scaleapi/vero.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC