The research group of Professor Zhuang Bohan at the State Key Laboratory of CAD&CG, Zhejiang University, presented an online paper titled “World R1: Reinforcing 3D Constraints for Text to Video Generation” at ICML 2026. This work introduces an innovative reinforcement learning framework that leverages pre trained 3D foundation models and vision language models as reward signals for post training alignment of text to video models. Without modifying the model architecture or introducing inference time 3D control modules, the proposed World R1 framework significantly enhances the 3D consistency of generated videos. It effectively upgrades general purpose video foundation models into geometry aware world simulators, offering a fresh design paradigm for large scale, scalable world generation systems.
Background
Text to video generation has broad applications in virtual reality, autonomous driving, embodied AI, and immersive content creation. Recently, a key sub challenge has emerged: enabling video foundation models to maintain stable geometric consistency under large camera motions and long range scenes. However, this task remains difficult because these models are inherently designed for 2D image space and lack an intrinsic understanding of real world 3D geometry.
Emerging 3D aware video generation methods offer a potential remedy by injecting explicit 3D representations into the video generation pipeline, thereby providing stronger geometric constraints. Pioneering approaches enforce that generated videos adhere to static 3D priors during inference, directly using 3D representations to constrain outputs. Yet these methods incur substantial additional computation at inference time, which limits the dynamic diversity and generalisation capability of the resulting videos. To address this, another line of work introduces extra 3D decoders or multi task learning frameworks that structurally modify pre trained video models and train them on static 3D aware datasets. Despite improvements, these approaches suffer from architectural complexity, limited applicability (often restricted to image to video tasks), and constrained dynamic adaptability. Moreover, explicit 3D modelling per video typically introduces high inference latency, severely hampering generation speed and efficiency. This raises a critical question: Can we design a general post training framework that awakens the latent geometric priors already present in video foundation models, without altering model architecture or adding inference time 3D modules, while preserving visual quality?
To answer this, recent advances in visual reinforcement learning—exemplified by Flow GRPO—have introduced various discriminative feedback mechanisms that directly align generative model outputs with human preferences. The aligned models then generate content that better matches human expectations, supervised by comparative reward scores. Reinforcement learning has increasingly become the dominant paradigm for post training alignment of generative models, thanks to its superior alignment performance and flexibility—it does not require retraining large models from scratch, gradually supplanting pure supervised fine tuning. Nevertheless, applying reinforcement learning directly to text to video generation for improving 3D consistency remains challenging due to the high dimensional nature of video generation space and the scarcity of 3D consistency feedback, often resulting in geometric hallucinations or suppressed dynamics in the generated videos.
overview of the proposed framework
Innovations and Contributions
This study proposes World R1, an innovative reinforcement learning framework for 3D consistency that enables 3D constraint alignment for text to video generation without modifying the base model architecture. World R1 features two key innovations. First, whereas conventional methods typically rely on incorporating additional 3D representation modules into video models and imposing geometric constraints at inference, we design a novel implicit camera conditioning and 3D aware reward mechanism. Here, camera trajectories are directly encoded into the initial latent noise, while pre trained 3D foundation models and vision language models provide discriminative feedback during training. This framework establishes an alignment between generated videos and real world 3D geometry, enabling efficient optimisation of the video model’s policy distribution via Flow GRPO to support scalable world simulation. Second, there is an inherent tension between rigid geometric constraints and dynamic scene fluidity—previous methods that pursued 3D consistency alone often over constrained motions, suppressing non rigid dynamics. To address this, we introduce a simple yet effective periodic decoupling training strategy that temporarily disables 3D rewards at fixed intervals, training only on a dynamic subset using generic quality rewards. This significantly balances geometric stability with dynamic diversity. Furthermore, we construct a pure text dataset oriented toward world simulation—rather than relying on 3D asset datasets—as the core of our post training pipeline. This not only decouples physical learning from visual bias but also ensures broad coverage of diverse scenarios.
Our main contributions are:
An innovative reinforcement learning framework for 3D consistency (World R1) that generates high quality, 3D consistent videos from text without modifying the base model architecture;
A comprehensive reward system combining 3D foundation models and vision language models, optimised via Flow GRPO to provide discriminative geometric feedback to video models;
A pure text dataset for world simulation, featuring multi category, multi level camera control, designed for post training alignment of video generation models.
Results and Contributions
Our method generates high quality videos with strong 3D consistency from text inputs. Below we present qualitative results and comparisons with other approaches.
Qualitative comparisons of generated videos show that our method significantly outperforms recent state of the art approaches on 3D consistency metrics.
Side by side comparisons with other methods demonstrate that our generated videos not only achieve superior geometric consistency but also preserve rich dynamic diversity.
compared to other methods, our approach generates videos that achieve both geometric consistency and well preserved dynamic diversity
Acknowledgements
Professor Zhuang Bohan from the State Key Laboratory of CAD&CG, Zhejiang University, serves as the corresponding author of this paper, with Ph.D. candidate Wang Weijie as the first author.
