Title: Decoupling Answer Engine Optimization from Platform Expansion: A Log-Driven Natural Experiment on ChatGPT Referral Traffic

Abstract

"Answer engines" powered by large language models (LLMs), such as ChatGPT, are now driving significant referral traffic to the open web. In response, a discipline similar to search engine optimization—termed Answer Engine Optimization (AEO)—has gained traction. While public case studies of AEO often highlight massive raw growth figures, these metrics are often skewed by the rapid overall expansion of the answer engine platforms themselves. To address this, we present a longitudinal field study focusing on a single high-traffic domain, glasp.co. This site hosts hundreds of thousands of YouTube question-and-answer pages and underwent a specific set of AEO interventions in January 2026 (outlined in Section 4).

By isolating the treatment to a subset of the site, the remaining untreated pages within the same domain serve as a contemporaneous control, effectively absorbing the broader platform growth trends. Utilizing first-party analytics and server logs instead of probabilistic third-party estimates, our analysis reveals four key findings:

1. Platform Growth Dominates Raw Metrics: The overall increase in raw referrals is largely driven by the platform’s natural tailwind. Over the study period, total monthly ChatGPT referrals increased by 5.7 times, whereas untreated pages on the same domain saw a 3.5 times increase.
2. Significant Intervention Effect: An interrupted time-series model analyzing the weekly treated-to-control ratio indicates a discrete level increase of 1.82x aligned with the interventions (95% CI 1.31–2.54, HAC p=0.001). This result remains robust when filtering for engagement and under alternative specifications, with engagement-filtered traffic showing a 2.27x increase.
3. Statistical Caution: Despite the initial findings, a conservative placebo-in-time permutation test yields a p-value of 0.16. Due to a short and noisy pre-intervention period, the effect is considered suggestive rather than conclusive.
4. No Negative SEO Impact: Google organic clicks to the treated pages did not decline beyond the site’s general trend, and indexation was maintained, aligning with the principle that AEO does not harm traditional SEO.

The primary methodological contribution of this study is the technique of separating treatment effects from platform tailwinds using an on-domain control group. This approach suggests that the headline growth multiples often cited in AEO success stories significantly overstate the actual causal impact of the optimization techniques.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC