Title: HypothesisMed: Structured Hypothesis-Space Reporting and Inference-Time Answer Fusion for Biomedical QA

Abstract:

While answer accuracy remains the standard metric for evaluating large language models in biomedical question answering, this single measure fails to capture critical dimensions such as output parseability, adherence to structured reliability protocols, the ability to identify weak answer spaces, or the avoidance of confident but incorrect assertions. To address these gaps, this study introduces HypothesisMed, a reliability pipeline designed for inference-time biomedical multiple-choice question answering. This framework integrates direct prompting, chain-of-thought reasoning, HypothesisMed-v3 prompting, and answer fusion mechanisms.

Within this system, answer fusion determines the final selection, while the HypothesisMed-v3 component generates SPACE labels alongside confidence metrics. These labels categorize the answer space as VALID, INCOMPLETE, or CONTRADICTED. The pipeline was tested on four models—Qwen2.5-7B, Phi-4-mini, DeepSeek-R1-32B, and BioMistral-7B—across the MedQA, MedMCQA, and PubMedQA datasets, utilizing 1,000 examples per dataset. Results indicate that the pipeline enhances weighted accuracy beyond each model’s optimal direct or chain-of-thought baseline, while simultaneously boosting both parse coverage and SPACE coverage.

Further evaluation scaled to 10,183 examples per model for Qwen2.5-7B and Phi-4-mini. In this larger-scale test, fusion techniques raised Phi-4-mini’s accuracy from 0.4296 to 0.5192. Although Qwen2.5-7B’s chain-of-thought approach maintained a marginal lead in raw answer accuracy, the fusion method achieved full parse and SPACE coverage with significantly reduced false commitments. A stress test involving 12,000 examples highlighted the persistent difficulty of answer-space diagnosis, yielding SPACE accuracies of 0.3074 for Qwen2.5-7B and 0.4168 for Phi-4-mini.

These findings demonstrate that answer accuracy, parseability, structured reliability reporting, calibration behavior, and false-commitment behavior constitute separable capabilities. Rather than claiming a universal state-of-the-art achievement, the primary contribution of this work is a reproducible inference-time framework. This framework enables the evaluation of biomedical question-answering models as auditable workflow components operating under structured reliability constraints.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC