SQuad accelerates video generation in Wan 2.2, reducing attention by 67x

The SQuad authors presented a method for distilling sub-quadratic attention for Video Diffusion Transformers. Instead of training a model from scratch, they distill a ready-made full-size Wan 2.2 5B model in two stages: Flow-Matching Supervised Fine-Tuning and Distribution Matching Distillation. The result is quality matching the teacher on VBench while reducing per-step per-block attention FLOPs by roughly 67x and halving end-to-end DiT latency. Details are in arXiv. The authors did not release the code.

What it means

This news relates to the text-to-video direction in general — see the video generators section. Among the tools in our reference, the direct object of the work is Wan, but with a caveat: the SQuad authors experimented on Wan 2.2 5B, whereas our fact sheet records the Wan 2.1 T2V-14B model. What Wan currently has: open source code and weights under Apache-2.0, works via diffusers, the largest weight file is 10.6 GB, all files total 64.3 GB. As described by the developer, the T2V-1.3B model requires 8.19 GB of VRAM and generates a 5-second 480P video on an RTX 4090 in about 4 minutes without optimizations like quantization. What is missing from the fact sheet: data on Wan 2.2 5B, on which the SQuad results were obtained, and information on when and in what form the distillation method might make it into public Wan releases. The SQuad developer does not specify whether the code or weights of the distilled model will be published. For now, SQuad remains a research result: the code is unavailable, and reproducing the method from the paper description is a separate engineering task. For practical use with Wan from our reference, there are no changes.