RefAlign: video generator — what it does and what you need to run it

RefAlign is an open-source video generator that creates clips from a text description and a reference image. The task the tool solves is image-text-to-video: an image and text go in, and a video consistent with both sources comes out. The authors describe the project as the official implementation of the RefAlign method for reference-based video generation, presented at ECCV 2026.

What it does

As described by the author, RefAlign solves the task of reference-to-video generation — generating video grounded in a reference. It is built on representation alignment: aligning representations between the reference image and the generated video. This makes it possible to preserve the visual characteristics of the source object or scene across the frames of the clip.

The repository is tagged with labels that point to the supported scenarios: subject-to-video (transferring a specific subject into a video), controllable-generation, text-image-generation and video-generation. Diffusion-models are also mentioned as the base architecture. The authors do not specify any particular limits on clip length, resolution or supported input image formats.

What you need to run it

The code is written in Python and published in the repository github.com/gudaochangsheng/RefAlign. The last code change is dated August 25, 2026; the repository was created on March 17, 2026. The code is distributed under the Apache License 2.0: its terms allow you to freely use, modify, distribute and sell the software, including for commercial purposes, provided you attach a copy of the license, keep the authorship notices and state the changes you made.

The model weights are hosted at huggingface.co/gudaochangsheng/RefAlign-1.3B. The total size of all weight files is 2.6 GB, and the largest single file also takes up 2.6 GB. The weights license is MIT. The authors do not specify VRAM requirements, system RAM requirements or particular library versions.

Who it suits

The tool will be useful to those working with reference-based video generation: you need to transfer a specific character, object or style from an image into a video sequence. The open source code and available weights let you study the implementation of the method and adapt it to your own tasks.

The project is unlikely to suit those looking for a ready-made application with an interface: the fact sheet describes code and model weights, not a user-facing product. The code is distributed under the Apache License 2.0, which permits commercial use subject to the distribution conditions.

RefAlign is a research implementation of a reference-based video generation method with open source code and published weights. Before running it, you should set up an environment for working with a Python project.