RefAlign: Video Generator — What It Does and What You Need to Run It

RefAlign is an open-source video generator that creates clips from a text description and a reference image. The task the tool addresses is image-text-to-video: an image and text go in, and a video consistent with both sources comes out. The authors describe the project as the official implementation of the RefAlign method for reference-based video generation, presented at ECCV 2026.

What it does

As described by the author, RefAlign addresses reference-to-video generation — generating video grounded in a reference. It is based on a representation alignment approach: aligning representations between the reference image and the generated video. This makes it possible to preserve the visual characteristics of the source object or scene across the frames of the clip.

The repository is tagged with labels that point to the supported scenarios: subject-to-video (transferring a specific subject into a video), controllable-generation, text-image-generation, and video-generation. Diffusion-models are also mentioned as the underlying architecture. The authors do not specify particular limits on clip length, resolution, or supported input image formats.

What you need to run it

The code is written in Python and published in the repository github.com/gudaochangsheng/RefAlign. The last code change is dated August 25, 2026; the repository was created on March 17, 2026. The code license is non-standard — the authors do not include its text in the fact sheet, so you need to check the terms manually before use.

The model weights are hosted at huggingface.co/gudaochangsheng/RefAlign-1.3B. The total size of all weight files is 2.6 GB, and the largest single file also takes up 2.6 GB. The weights license is MIT. The authors do not specify VRAM requirements, system RAM requirements, or particular library versions.

Who it suits

The tool will be useful to those working with reference-based video generation: transferring a specific character, object, or style from an image into a video sequence. The open source code and available weights make it possible to study the implementation of the method and adapt it to your own tasks.

The project is unlikely to suit those looking for a ready-made application with an interface: the fact sheet describes code and model weights, not a user-facing product. You should also keep in mind the non-standard code license — if its terms turn out to be restrictive, use in a commercial project may require separate clearance.

RefAlign is a research implementation of a reference-based video generation method with open source code and published weights. Before running it, check the code license and prepare an environment for working with a Python project.