Title: EvoPool: Evolutionary Programmatic Annotation for Label-Efficient Specialized Supervision

Abstract:

While large language models demonstrate superior performance in general-purpose tasks, they often lag behind smaller, supervised models in specialized, high-stakes fields where acquiring training labels is expensive. To tackle this challenge, we introduce EvoPool, a multi-agent framework driven by principles of Darwinian evolution. In this system, three distinct agents iteratively generate executable code for annotators. A small validation set serves as the fitness signal, while a deterministic gate ensures that only annotators meeting specific criteria for viability, diversity, and marginal contribution are retained across generations.

The resulting pool of votes is converted into soft training labels through EvoAgg, an aggregator that integrates semantic features with annotator-vote data. This proprietary pool operates at nearly zero cost per example and achieves speeds 4,500 to 31,000 times faster than LLM-based annotation when processing 100,000 examples. In evaluations across eight complex, specialized tasks where LLMs typically struggle—including biomedical relation extraction, legal clause classification, complex reasoning, and dense multi-label biomedical classification—EvoPool outperformed the strongest LLM annotation baseline by an average of +0.141 in macro-F1. Notable improvements included a +0.301 gain on ChemProt and a +0.265 gain on PubMed. The source code is publicly accessible at: https://github.com/tianyi0216/EvoPool

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC