Title: Verifying Bit-Exact AI Inference Without Sacrificing Performance

Abstract: Establishing trust in AI governance against covert adversaries—entities that adhere to monitoring protocols only when the risk of detection is high—requires the ability to verify AI workloads with certainty. However, the inherent non-determinism of GPU floating-point arithmetic typically compels auditors to settle for approximate output matches. This limitation allows covert actors to exploit unverified variables within monitored computations, facilitating attack vectors such as steganography, the undetected alteration of inference software, and hidden processing through unreported batch elements.

In this study, we empirically examine how contemporary inference engines, specifically vLLM and Hugging Face Transformers, generate deterministic yet non-invariant outputs. Crucially, this occurs without the need to enable performance-degrading determinism flags, provided that sufficient information is available for re-computation and no atomic functions are invoked in the backend. We further demonstrate that bitwise-precise re-computation is achievable without access to identical hardware. By employing a software-only emulation of LLM inference across various NVIDIA GPU architectures, we show that accumulated rounding errors can serve as an auditable signature of the specific software and hardware configuration used during inference, thereby transforming a perceived limitation into a viable verification mechanism.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC