Title: CARVE: Utilizing Resampling for Clustering Validation and Exploration

Abstract: Clustering serves as a fundamental tool for enabling data-driven scientific breakthroughs across various disciplines. Nevertheless, the outcomes of clustering are notoriously volatile, heavily influenced by decisions regarding algorithm selection, data preprocessing, and the determination of the cluster count ($k$). This sensitivity often leads to scientific findings that lack reproducibility. While the prevailing method for assessing clustering quality involves Clustering Validation Indices (CVIs)—such as the Silhouette score, Davies-Bouldin index, and Calinski-Harabasz index—these metrics depend on geometric assumptions that fail when applied to the complex, heavy-tailed, high-dimensional, and nonlinear data typical in biomedical studies. Although resampling-based approaches, which focus on clustering stability and generalizability, have been suggested, they currently exist in fragmented, specialized tools lacking a unified, user-friendly software solution. To address this deficiency, we introduce CARVE (Cluster Analysis with Resampling for Validation and Exploration), an open-source package available in both Python and R. CARVE simultaneously assesses various clustering algorithms and hyperparameters, providing diagnostics on stability and generalizability at the global, cluster, and individual sample levels. It also offers principled selection guidelines and generates consensus-based cluster labels. In tests involving six synthetic benchmarks, CARVE consistently identified near-optimal clusterings, whereas traditional indices performed poorly. Furthermore, when applied to experimental genomics and proteomics datasets, CARVE successfully uncovered finer biological structures even in scenarios where classical CVIs failed completely. The tool features a Python API compatible with scikit-learn and an R interface designed to integrate seamlessly with Seurat workflows.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC