A research paper titled “Sparse-to-Complete: From Sparse Image Captures to Complete 3D Scenes” by Research Professor Shao Tianjia’s group at the State Key Laboratory of Computer Aided Design and Computer Graphics, Zhejiang University, has been published at SIGGRAPH 2026. The work addresses key challenges in sparse-view 3D reconstruction—few input images, insufficient scene coverage, and incomplete reconstruction—and proposes a new sparse-view 3D scene reconstruction framework called S2C-3D. With only 6 to 8 images, the method generates high-fidelity, complete, and multi-view consistent 3D scene representations. The team combines diffusion model image inpainting with 3D Gaussian representations, and introduces a training-free multi-view consistency conditional sampling process that mitigates view conflicts among diffusion-inpainted images at the source. It further incorporates scene-specific diffusion model fine-tuning and camera trajectory planning that covers the full scene to improve inpainting quality and reconstruction completeness. Compared with existing methods, S2C-3D effectively reduces blur, artifacts, and missing regions common in sparse-view reconstruction, offering a new technical path for low-cost 3D content capture and generation.
Research Background
High-quality 3D scene reconstruction is valuable for virtual reality, augmented reality, digital film production, robot perception, and autonomous driving. In recent years, methods such as NeRF and 3D Gaussian Splatting (3DGS) have significantly improved 3D scene reconstruction and novel view synthesis. However, these methods typically rely on large numbers of multi-view images, making capture time-consuming and demanding on user skill. To reduce capture cost, sparse-view 3D reconstruction has become a research hotspot. Existing methods typically initialize a low-quality 3D representation from a few input images, then use image diffusion models to complete renderings from unknown views, and use the inpainted images as supervision to optimize the low-quality 3D Gaussians. However, under extremely sparse input conditions, diffusion models must inpaint invisible regions from different unknown views independently. The content “imagined” by the generative model often lacks multi-view consistency and accumulates into blur and structural artifacts during 3D optimization. This multi-view consistency issue is a key bottleneck limiting sparse-view reconstruction quality. In addition, there is a distribution gap between general pretrained diffusion models and the target scene, which can produce inpaintings that do not match scene structure in large unobserved areas. Simple camera interpolation commonly used in existing methods also struggles to cover the full scene, leading to obvious missing regions in reconstructions.
Research Innovations and Breakthroughs
This work proposes S2C-3D, a complete 3D scene reconstruction framework for extremely sparse image inputs. Building on the paradigm of using image diffusion models to optimize low-quality 3D Gaussians, S2C-3D takes multi-view consistency of inpainted images as a core design goal, explicitly constraining content consistency across different virtual views during diffusion inpainting. Specifically, we propose a training-free multi-view consistency conditional sampling process. It first uses geometric projection to establish pixel-level mappings between adjacent inpainting views and measures pixel-level differences between inpainted contents across views. This consistency measure is then injected as a condition into the diffusion sampling process, guiding the model to generate multi-view consistent inpainted images. Finally, these multi-view consistent images are used to optimize the 3D Gaussian representation, effectively reducing blur and severe artifacts common in sparse-view reconstruction. Around this core contribution, S2C-3D introduces two key modules. First, we build a scene-specific diffusion model by fine-tuning on input images and their corresponding degraded images, adapting the diffusion model to the current scene distribution and reducing the gap between general diffusion priors and the target scene. Second, we design an information-gain-based camera trajectory planning method that actively selects virtual camera positions yielding more visible information based on a scene coverage metric, rather than simply interpolating between input views, providing more complete supervision views for 3D Gaussian optimization. In summary, this work makes the following contributions:
Proposes S2C-3D, a sparse-view 3D scene reconstruction framework that achieves high-quality complete scene reconstruction from only 6 to 8 input images.
Proposes a training-free multi-view consistency conditional sampling process that injects multi-view consistency constraints into diffusion sampling, effectively reducing conflicts between different inpainting views and improving the stability and visual quality of the final 3D representation.
Designs a scene-specific diffusion model that improves inpainting quality through lightweight fine-tuning for the target scene, mitigating the distribution gap between general diffusion models and the target scene.
Figure: System framework of the method.
Proposes an information-gain-based camera trajectory planning method that enables virtual cameras to cover more unobserved areas, improving complete scene reconstruction capability.
S2C-3D achieves 3D scene reconstruction with scene completeness, visual fidelity, and multi-view consistency under extremely sparse image inputs, and demonstrates high stability and generalization across multiple datasets. Below are 3D reconstruction results generated by our method.
The figure above shows quantitative experimental results. Overall, S2C-3D outperforms existing baseline methods in sparse-view scene reconstruction, indicating that it can reconstruct higher-quality 3D scenes
Research Professor Shao Tianjia of the State Key Laboratory of Computer Aided Design and Computer Graphics, Zhejiang University, is the corresponding author. PhD student Shen Yiyang is the first author. This work was supported by the National Natural Science Foundation of China (Grant Nos. U23A20311, 62322209, and 62421003) and the XPLORER PRIZE.
