object-permanence: video generator — what it does and what you need to run it

object-permanence is an open source video generator that solves the video-to-video task. The authors describe it as a codebase for training object permanence in world models. The project was created by developer Hokin Deng and is aimed at working with video and scenes where objects must persist between frames.

What it does

As described by the author, the project is intended for training object permanence in world models. This means the generator takes video as input and outputs video, keeping objects consistent between frames. The repository lists tags that give an idea of the project's focus: blender, object-permanence, trainium, video-generation and world-model. The authors do not provide a detailed list of supported scenarios or restrictions on input video types.

What you need to run it

The code is written in Python and uses the cosmos library. The repository is open, the source code is available at github.com/hokindeng/object-permanence, the model is published at huggingface.co/Hokin/PWM-WROP, and the project website is object-permanence.world.

Running it requires space for the weight files: the largest file is 4.7 GB, and all weight files together are 28.2 GB. The authors do not specify requirements for system RAM, GPU or a particular operating system.

The code license is CC BY-NC 4.0 with an additional OpenMDW-1.1 agreement. The repository, including the code and the PWM-WROP model, may be used, modified and distributed for non-commercial purposes only, with attribution and preservation of the license. Parts based on NVIDIA Cosmos, including the base model and adapted code, are distributed under OpenMDW-1.1 without restrictions, including commercial use, provided the text of the agreement and copyrights are preserved. The model weights are licensed under CC BY-NC 4.0.

Who it suits

The project will be useful to those doing research in video generation and world models, especially if the task involves preserving objects between frames. The open source code and published weights make it possible to study the implementation and adapt it for your own experiments.

The project is not suitable for those looking for a ready-made commercial production tool: the main part of the code and the PWM-WROP model are limited to non-commercial use. It is also worth noting that the authors do not provide hardware requirements, so before running it you will have to assess on your own whether you have enough resources to work with the model.

object-permanence is a research codebase with open source code focused on the task of object permanence in video. The project provides access to the code and weights, but most of it is intended for non-commercial use, and the authors do not detail the requirements for running it.