Title: The Importance of Context: A Repository-Aware Security Evaluation of the Agent Skill Landscape

Abstract:

Agent skills serve to augment the capabilities of local artificial intelligence agents, including tools like OpenClaw and Claude Code. As their adoption accelerates, the ecosystem has spawned dedicated marketplaces akin to mobile app stores, alongside automated scanning tools designed to distinguish between safe and harmful skills. However, recent reports from individual marketplaces have flagged as many as 46.8% of skills as malicious, prompting significant concern regarding the prevalence of false positives.

This study presents the most comprehensive empirical security analysis of the AI agent skill ecosystem conducted to date. By aggregating data from GitHub and three primary distribution platforms, we gathered and examined 238,180 distinct skills, focusing on their behavioral patterns, internal composition, and repository environments. In contrast to traditional scanner-based methods that typically evaluate skills in isolation, our approach incorporates a repository-aware perspective. This method verifies whether a flagged skill aligns with the broader context of its associated GitHub project.

The inclusion of this contextual data dramatically lowers the count of suspicious items; after applying repository-aware analysis, the rate of suspicious skills drops to just 0.52%. These findings indicate that current scanning tools may significantly overstate the level of threat when repository context is disregarded. Nevertheless, our investigation also uncovered undocumented, real-world attack vectors, such as the exploitation of skills hosted within abandoned GitHub repositories. Ultimately, these insights offer a more accurate assessment of the risk landscape within the agent-skill ecosystem and underscore the necessity for security evaluations that account for contextual information.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC