Title: Scaling PEFT: The Path to Million-Scale Personal Models with Trillion-Parameter Foundations

Abstract:

While Parameter-Efficient Fine-Tuning (PEFT) is traditionally viewed merely as a cost-effective substitute for full model fine-tuning, this work explores its broader potential. We propose treating small, trainable adapters as persistent local state layers atop robust, shared foundation models. In this conceptual framework, the base model supplies general competence, while the adapters encode instance-specific behaviors, including user preferences, specialized skills, tool usage habits, and memory-like updates.

We structure our analysis around three distinct scaling dimensions: 1. Scale Up: Investigating how stronger shared priors enhance the utility of small local updates. 2. Scale Down: Examining the limits of adapter size while maintaining reliability. 3. Scale Out: Managing the coexistence of numerous persistent, adapted instances.

As an illustrative example of the required infrastructure, we introduce MinT, which handles adapter identity, versioning, provenance, evaluation, and serving residency. Our findings indicate that PEFT can serve as a compact substrate for persistent personal models, extending far beyond its role as a simple budget alternative to full fine-tuning.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC