OctWorld

Long-range generative world modeling

OctWorld: Long-Range World-Consistent Video Generation with Octree-based 3D Mapping

Zelong Lv1,2,* Sicheng Xu2 Jianfeng Xiang2,3,* Ruicheng Wang1,2,* Yue Dong2 Yu Deng2 Guangzhong Sun1 Jiaolong Yang2,†

1 University of Science and Technology of China   2 Microsoft Research Asia   3 Tsinghua University

Read Research Paper Code (Coming Soon)

01 / Visual overview

See OctWorld in motion.

OctMap-Guided Generation with Precise Camera Control

A system-level visualization of the OctWorld generation workflow, showing how data and conditioning signals flow among camera control, RGB-D generation, OctMap memory, and video diffusion components to enable precise control along user-specified camera trajectories.

Inside OctMap Online Integration

A detailed visualization of the OctMap subsystem, showing how incoming RGB-D observations are unprojected, fused into adaptive octree nodes, and integrated online to update the scene's geometry and appearance throughout generation.

03 / Abstract

Explorable worlds with persistent 3D memory.

We present OctWorld, a video diffusion framework with 3D memory for generating explorable, world-consistent, and high-fidelity visual scenes. Given a single image, OctWorld enables stable autoregressive world generation along user-specified camera trajectories.

We focus on the problem of long spatial range generation, characterized by long camera path and wide viewing angle coverage, where maintaining spatial consistency, particularly when revisiting previously generated regions, becomes challenging. To address this, we first build upon a pretrained autoregressive video diffusion model that supports longer-horizon context modeling. We adapt this model to RGB-D generation and introduce a depth normalization scheme for inter-chunk consistency.

Next, and more importantly, we propose an extensible and spatially-adaptive 3D memory, termed OctMap. OctMap constructs a global 3D representation of the generated scene by progressively fusing the generated RGB-D frames. To accommodate diverse scene scales in the open world setting, it applies adaptive spatial resolution dynamically determined by image evidence with a sparse octree hierarchy, which preserves effective geometric and appearance details while maintaining low memory overhead.

Experiments demonstrate that OctWorld can generate long-range and spatially consistent videos, outperforming prior methods on existing benchmarks as well as in longer-range generation cases.

04 / Method

A persistent map for worlds that keep unfolding.

OctWorld training pipeline showing history and target RGB-D frames, OctMap construction, spatial conditioning, and diffusion training.
Training pipeline of OctWorld. Given a training RGB-D segment, we split the sequence into history frames and target frames. An OctMap is constructed from the history frames and rendered to the target view as spatial conditions. εα and εt denotes gaussian noises at level α and t respectively.

05 / Citation

Reference OctWorld.

@inproceedings{lv2026octworld,
  title     = {OctWorld: Long-Range World-Consistent Video Generation
               with Octree-based 3D Mapping},
  author    = {Lv, Zelong and Xu, Sicheng and Xiang, Jianfeng and
               Wang, Ruicheng and Dong, Yue and Deng, Yu and
               Sun, Guangzhong and Yang, Jiaolong},
  year      = {2026}
}