Programmable World Model — a new framework for controllable video worlds with persistent state

A paper titled Programmable World Model has been published on arXiv, in which the authors propose a framework for creating interactive video worlds with an explicit state that persists across frames. Instead of generating video directly from a text description, the system translates natural language instructions into executable programs describing entities and the rules of their interaction. A lightweight engine maintains the global world state, including off-screen objects and non-visual attributes. To link the state to video generation, an intermediate representation is used — state-augmented 3D oriented bounding boxes (OBB), which, together with the camera trajectory, are compiled into conditioning signals for a pretrained video model. The authors also presented the CombatStateBench benchmark, on which their method achieves 94% object counting accuracy and 98% state accuracy. The code has been released as open source.

What it means

This news note concerns the field of video generation in general — our AI Video section gathers tools that tackle the task differently. Programmable World Model is not a video generator in the usual sense, but a framework on top of a pretrained video model: it adds a state management layer and rules, while the video model acts as the renderer. There is no direct equivalent among our tool pages. The closest tools on the topic from the reference:

• Lance — an open any-to-any model from Bytedance with 3B active parameters, weights take up 28.6 GB, the largest file is 13.2 GB. The authors require a GPU with 40 GB of VRAM and CUDA 12.4+. The model can understand and generate images and video, but there is no persistent world state or programmable rules in its fact sheet.
• LTX-Video — an image-to-video generator from Lightricks, weights 236.4 GB, code license Apache-2.0, weights license — other. It claims to work with 1 GB of VRAM and supports MPS on macOS. This is a classic short-clip generator with no state mechanics.
• CogVideoX — a text-to-video model from Zhipu AI, weights 20 GB, code license Apache-2.0, weights license — other. It requires CUDA, and the authors mention NVIDIA H100. It also does not support a persistent world state.

Programmable World Model solves a problem that is absent from the fact sheets of our tools: explicit entity management, transition rules, and state persistence beyond the visible frame. For a user choosing a tool in our reference, this is more a signal about the direction of development than a ready-made tool for running on your own hardware: the authors do not specify VRAM requirements, weight size, or license in the paper description.