Title: Study Protocol: How Configuring Agentic AI Coding Tools Influences Build-vs-Buy Decisions

Abstract

Agentic AI coding assistants are gaining autonomy in code generation, which inherently involves choices between importing existing libraries and writing functionality from scratch. These "build-versus-buy" determinations significantly impact software security, licensing adherence, performance metrics, and long-term maintainability. Despite their importance, no controlled experimental research has yet investigated the factors driving these decisions in agentic AI systems.

Configuration mechanisms—the methods developers use to customize tool behavior for specific projects or workflows—serve as a primary lever for influencing these outcomes. However, it remains unknown which specific configuration strategies most effectively guide build-versus-buy choices. To address this gap, we present a pre-registered study protocol designed to examine how configuration mechanisms alter build-versus-buy behaviors in two widely used tools: Claude Code and OpenAI Codex.

Our methodology involves executing controlled programming tasks derived from a benchmark of staged projects, each featuring identifiable build-versus-buy decision points. We will manipulate the configurations provided to each tool, testing a range of inputs from no configuration to context files containing soft preferences and explicit prohibitions, as well as Skills (autonomously discoverable instructions), MCP-enabled library discovery tools, and permission controls. The study will measure the libraries selected by the tools, assess whether newly introduced libraries are disclosed, and verify the completeness and accuracy of those disclosures.

The protocol is structured around nine pre-registered hypotheses. Upon completion, the resulting benchmark dataset and analysis pipeline will be released as a reusable artifact to facilitate future evaluations of build-versus-buy behavior in agentic AI coding tools.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC