Title: Beyond Access: Guided LLM Scaffolding for Independent Learning in Undergraduate Statistics

Abstract:

As large language models (LLMs) become ubiquitous in student learning environments, their true educational benefit hinges on whether they foster critical reasoning or merely facilitate task completion through disengaged use. This research investigates the efficacy of guided LLM integration within an undergraduate Probability and Statistics course, specifically addressing the disconnect between mere access to these tools and the quality of student interaction with them.

The study employed a four-week quasi-experimental design during a summer program, dividing participants into three equally balanced groups: one with no LLM access, one with unrestricted LLM access, and one with guided LLM access. While the guided and unrestricted groups utilized the same LLM platform, the guided cohort received specific instruction and protocols designed to encourage reasoning-centric help-seeking. These guidelines emphasized requesting stepwise hints, verifying results, and adhering to ethical standards. To isolate the impact of AI-supported practice from genuine independent mastery, all quizzes and a delayed final exam were administered without any LLM or external assistance.

The findings indicate that guided usage fostered more distinct, learning-oriented interaction patterns compared to unrestricted access, particularly in how students prioritized understanding concepts over obtaining final answers and sought incremental support. During the intervention period, students in the guided LLM condition demonstrated superior performance on quizzes completed without assistance. Conversely, unrestricted access seemed to aid in completing assisted tasks more than it did in consistently enhancing independent performance. Data on available time did not suggest that simple duration of use explained these outcomes. Furthermore, self-assessment calibration revealed that students in the guided condition exhibited a closer alignment between their perceived understanding and their actual demonstrated knowledge.

Ultimately, the study concludes that providing LLM access in isolation is an insufficient educational strategy. For the field of Artificial Intelligence in Education (AIED), the primary design imperative is to scaffold student interactions with LLMs, ensuring these systems serve as collaborative partners in reasoning rather than mere instruments for acquiring answers.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC