Title: HLL: Can AI Agents Surpass Humanity’s Final Verification Barrier?

Abstract:

As multimodal agents are increasingly tasked with managing user interfaces on behalf of individuals, a critical deployment challenge has emerged: can these systems genuinely replace humans in workflows that services intentionally shield from automation? CAPTCHA verification brings this issue into sharp focus. More than just a visual riddle, it serves as a strict human-verification checkpoint preceding sensitive actions such as account registration, content access, and form submission.

To address this, we present Humanity’s Last Line of Verification (HLL), a controlled benchmark designed to test whether agents can navigate this boundary via grounded, human-like interaction rather than relying solely on object recognition. HLL encompasses a wide array of CAPTCHA interaction types and subjects agents to controlled realism stressors, including cluttered webpage layouts, more difficult task variations, and validation processes conditioned on action traces.

We assessed eight leading frontier multimodal agents within a closed-loop GUI environment. The findings indicate that current agents remain fragile at this threshold of human substitution. Performance fluctuates significantly depending on the verification type, deteriorates under realistic interface conditions, and falls further when agents are required to justify correct answers with valid action traces. By highlighting deficiencies in localization, action calibration, state tracking, and process consistency, HLL offers a tangible testbed for gauging how close multimodal agents are to functioning as human substitutes in secure, real-world workflows.

Our code is available at https://github.com/XinhaoS0101/HLL

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC