Title: Achieving Expert-Level Chip Placement Through Reinforcement Learning

Physical design relies heavily on chip placement, a crucial phase where recent reinforcement learning (RL) techniques have shown promise. However, these methods typically prioritize wirelength optimization during training, which frequently results in layouts that fall short of expert quality. We pinpoint the design of the reward function as the main driver behind this performance deficit. Rather than attempting to mathematically model complex design processes, we bypass this hurdle by deriving a reward model directly from expert layouts.

Our methodology begins with final expert designs to reconstruct step-by-step expert trajectories. These trajectories serve as either demonstrations or preference data to train a model capable of capturing the implicit, latent rewards embedded within expert outcomes. Our experimental results demonstrate that the proposed framework can effectively learn from as little as one design while maintaining strong generalization capabilities for previously unseen scenarios.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC