Programmable World Model — a new framework for controllable video worlds with persistent state

A paper titled Programmable World Model has been published on arXiv, in which the authors propose a framework for creating interactive video worlds with an explicit state that persists across frames. Instead of generating video directly from a text description, the system translates natural-language instructions into executable programs that describe entities and the rules of their interaction. A lightweight engine maintains the global world state, including off-screen objects and non-visual attributes. To link the state to video generation, an intermediate representation is used — state-augmented 3D oriented bounding boxes (OBB), which, together with the camera trajectory, are compiled into conditioning signals for a pretrained video model. The authors also introduced the CombatStateBench benchmark, on which their method achieves 94% object-counting accuracy and 98% state accuracy. The code has been released as open source.

What it means

This news note concerns the field of video generation in general — our AI Video section gathers tools that tackle the task differently. Programmable World Model is not a video generator in the usual sense, but a framework on top of a pretrained video model: it adds a state-management layer and rules, while the video model acts as the renderer. There is no direct equivalent among our tool pages. The closest related tools from the reference:

• Lance — an open any-to-any model from Bytedance with 3B active parameters, weights take up 28.6 GB, the largest file is 13.2 GB. The authors require a GPU with 40 GB of VRAM and CUDA 12.4+. The model can understand and generate images and video, but its fact sheet has no persistent world state or programmable rules.
• LTX-Video — an image-to-video generator from Lightricks, weights 236.4 GB, code license Apache-2.0, weights license — LTXV Open Weights License 0.X: commercial use is limited by a revenue threshold. It claims to work with 1 GB of VRAM and supports MPS on macOS. This is a classic short-clip generator with no state mechanics.
• CogVideoX — a text-to-video model from Zhipu AI, weights 20 GB, code license Apache-2.0, weights license — The CogVideoX License: commercial use after registration, up to 1 million visits per month. It requires CUDA, and the authors mention NVIDIA H100. It also does not support a persistent world state.

Programmable World Model solves a problem that is absent from our tools' fact sheets: explicit entity management, transition rules and state persistence beyond the visible frame. For a user choosing a tool in our reference, this is more a signal about the direction of development than a ready tool for running on your own hardware: the authors do not specify VRAM requirements, weight size or license in the paper description.