Title: Tangle Cores for Topology-Aware State Abstraction in Markov Decision Processes

Abstract

Reinforcement learning typically defines state abstraction by partitioning states according to similarities in rewards and transitions. However, this approach overlooks a prevalent structural characteristic in navigation, graph, and hierarchical decision-making tasks: interface elements like hubs, doors, and bottlenecks often belong to multiple regions simultaneously. To address this, we propose tangle-core abstraction, an overlapping state-abstraction framework grounded in the graph tangles of empirical transition graphs. This technique generates abstract states from consistently oriented low-order separations and utilizes a membership kernel to represent shared interfaces, moving away from rigid partitions. We establish value-preservation guarantees for the resulting overlapping abstract MDP, contingent upon an explicit action-consistency condition. Furthermore, we derive an error decomposition formula highlighting interior homogeneity versus boundary leakage and provide a quantitative analysis of interface overlap, demonstrating the conditions under which hard partitions lead to unnecessary boundary errors. Experimental evaluations across bottlenecked tabular domains, procedurally generated mazes, and MiniGrid environments show that tangle-core abstractions deliver superior compression-to-return trade-offs compared to baselines including reward-aware methods, learned approaches, topological maps, and graph-partitioning techniques. Conversely, we identify a specific failure mode where transition topology lacks informativeness, in which case tangles yield minimal advantages. These findings establish graph tangles as a robust topology-aware abstraction prior for decision problems featuring shared interface structures.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC