Title: From Prompt to Service: An SLM-Based Agent Orchestration Gateway for AI-Driven Virtual Worlds

Original: arXiv:2606.03557v1 Announce Type: new Abstract: As generative AI capabilities expand, AI-driven virtual worlds face a growing architectural challenge. Users interact through in-world interfaces in multimodal ways, yet their requests demand fundamentally different AI backend models and computational resources. Embedding these capabilities directly into virtual world systems reduces extensibility, complicates maintenance, and limits the ability to coordinate services distributed across edge and cloud infrastructure. This paper presents an SLM-based Agent Orchestration Gateway, a lightweight runtime coordination mechanism that decouples a virtual world client from heterogeneous AI backends through intent-driven service routing. An edge-deployed SLM classifies the semantic intent of each user prompt, a configurable service registry validates and resolves the routing decision, and the selected backend is invoked transparently, enabling new AI capabilities to be introduced in the virtual world without modifying the client application. The gateway is implemented and evaluated within the InterwovenXR virtual museum testbed. The evaluation shows that compact SLMs can serve as reliable intent routers on edge hardware, and that task-specific fine-tuning can transform sub-billion-parameter models into practical, low-latency routers. A layered configuration pairing a fine-tuned sub billion-parameter model as router with a larger SLM for conversational response generation is shown to be deployable on mid-range edge hardware and more efficient than delegating both responsibilities to a single model. The findings show that SLMs can support practical AI service orchestration in virtual worlds and the work contributes an evaluated architecture for scalable, extensible, and edge-supported AI interaction, enabling virtual agents become access points to distributed generative AI services.

Rewrite: Title: Bridging Prompts and Services: An SLM-Driven Gateway for Orchestrating AI in Virtual Worlds

Original: arXiv:2606.03557v1 Announce Type: new

Abstract: The rapid advancement of generative AI has introduced significant architectural hurdles for AI-powered virtual environments. While users engage with these worlds through multimodal in-world interfaces, their diverse requests require distinct AI backend models and varying computational power. Integrating these functionalities directly into the virtual world’s core system hampers extensibility, complicates maintenance, and restricts the coordination of services spread across cloud and edge infrastructures. To address this, this study introduces an SLM-based Agent Orchestration Gateway, a streamlined runtime framework that separates virtual world clients from diverse AI backends via intent-based service routing. In this architecture, an SLM deployed at the edge interprets the semantic intent of user prompts, while a configurable service registry confirms and executes the routing logic. The chosen backend is then accessed seamlessly, allowing virtual worlds to adopt new AI features without requiring updates to the client software. We implemented and assessed this gateway using the InterwovenXR virtual museum as a testbed. Our results demonstrate that compact SLMs function effectively as intent routers on edge devices, and that targeted fine-tuning can elevate sub-billion-parameter models into efficient, low-latency routing tools. Furthermore, a hierarchical setup—utilizing a fine-tuned sub-billion-parameter model for routing alongside a larger SLM for conversational output—proves viable on mid-tier edge hardware and offers greater efficiency than assigning both tasks to a single model. These insights confirm that SLMs are capable of facilitating practical AI service orchestration within virtual worlds, contributing a validated architecture for scalable, extensible, and edge-enabled AI interactions that allow virtual agents to serve as gateways to distributed generative AI services.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC