Title: Navigating the Pitfalls and Solutions of Multi-Agent Debate in Data Cleaning

Abstract: This study investigates the dual nature of multi-agent debate in data cleaning, determining when it serves as a beneficial tool and when it becomes detrimental. Analyzing data across three distinct benchmarks, four model architectures, and more than 6,000 task-condition combinations, we observed a reversal in the efficacy of debate mechanisms. Specifically, debate was found to impair generation capabilities across all four tested models, causing performance drops ranging from 1.6 to 15.5 percentage points. This degradation stems from Critique-Induced Confusion (CIC), a phenomenon where Generators accept hallucinated feedback from Critics without sufficient scrutiny. Conversely, the debate framework significantly enhanced error detection, yielding a 27.4-point increase in F1 score with an effect size of d=1.0.

We established a theoretical condition for debate utility: it is advantageous when the likelihood of salvaging an incorrect output—calculated by weighting the probability of successful Critic verification against the output's fixability—surpasses the risk of compromising a correct output. Through factorial experimentation, we demonstrated that adversarial separation is a critical requirement for success. Systems relying on self-verification using identical tools proved ineffective. In contrast, a configuration featuring a distinct Critic equipped with code-execution grounding, combined with evidence-gated generation, achieved the first significant improvement over single-agent baselines in generative tasks (+5.3pp, p<0.05). Our derived condition accurately predicted outcomes for all nine task types and validated with zero false positives across 19 published comparisons spanning seven different domains.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC