Title: Efficient and Swift Novel View Synthesis Using Differentiable Multiplane Images

Original: arXiv:2606.02068v1 Announce Type: cross Abstract: In recent times, the field of novel view synthesis has seen significant advancements, with leading techniques like Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) achieving notable success. Nevertheless, these methods frequently face challenges in balancing rendering efficiency and model compactness, while their optimization-driven training processes are often excessively lengthy. Moreover, they generally depend on dense input data, which leads to suboptimal outcomes when only sparse viewpoints are available. While feed-forward reconstruction methods have notably decreased the training time for 3DGS, their pixel-level formulation creates millions of Gaussian points from a single image, thereby hindering their practical application on mobile platforms. To overcome these drawbacks, we return to the Multiplane Image (MPI) representation, a technique that depicts scenes through a streamlined set of planar layers to enable efficient novel view synthesis. By capitalizing on recent breakthroughs in visual foundation models, we employ predicted point maps to ensure robust geometric initialization, which is then refined through differentiable optimization. To mitigate the artifacts and gaps that arise from sparsely initialized MPIs, we incorporate a one-step diffusion process that integrates into both the differentiable MPI optimization and the post-rendering stage. When benchmarked against a prominent GS-based method, our solution operates 30.7% more quickly and utilizes merely 14.8% of the model size, all while maintaining comparable synthesis quality for front-view applications.

Rewrite: Recent years have brought substantial improvements to novel view synthesis, with dominant approaches such as 3D Gaussian Splatting (3DGS) and Neural Radiance Fields (NeRF) delivering strong performance. Despite these successes, these techniques often encounter difficulties in reconciling rendering speed with model size, and their reliance on optimization-based training can be prohibitively slow. Additionally, they typically require dense input data, leading to poor performance in sparse-view settings. Although feed-forward reconstruction has accelerated 3DGS training, its pixel-aligned nature generates millions of Gaussians per image, which restricts its viability for mobile deployment. To resolve these issues, we revisit the Multiplane Image (MPI) representation, which offers efficient novel view synthesis by modeling scenes with a compact series of planar layers. Building on recent developments in visual foundation models, we use predicted point maps for stable geometric initialization, followed by differentiable optimization. To tackle the holes and artifacts common in sparsely initialized MPIs, we introduce a one-step diffusion mechanism that contributes to both the differentiable optimization of the MPI and the post-processing of the rendered output. In comparison to a representative GS-based method, our approach is 30.7% faster and requires only 14.8% of the model size, while delivering competitive synthesis quality in front-view scenarios.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC