MMAudio: audio generator — what it does and what you need to run it

MMAudio is an open-source audio generator that synthesizes an audio track from video or a text description. The developers position it as a tool for high-quality sound synthesis with joint training on multiple data types. The code is published on GitHub, and the model weights are available on Hugging Face.

What it does

As described by the authors, MMAudio solves two main tasks:

• video-to-audio — generating a sound track for a given video;
• text-to-audio — synthesizing sound from a text description.

The authors place the project in the field of deep learning and computer vision. The repository description states that the work was accepted at CVPR 2025. The latest release is v0.1.

What you need to run it

The code is written in Python. As described by the author, Python 3.9 or newer and PyTorch 2.5.1 or newer with matching versions of torchvision and torchaudio are required. The platform is CUDA; the authors recommend choosing a PyTorch version that matches your CUDA version on the pytorch.org website.

VRAM as described by the author is about 6 GB in 16-bit mode. Quoted directly from the description: «In our experiments, inference only takes around 6GB of GPU memory (in 16-bit mode) which should fit in most modern GPUs». This is a developer claim, not an independent measurement.

The largest model weights file takes up 19.2 GB, and all weight files together total 50.5 GB. The code license is MIT, and the weights license is cc-by-nc-4.0. This means the weights can only be used for non-commercial purposes, while the code is free to use, including commercially, provided the license notice is retained.

Who it suits

The tool is designed for those working with sound generation for video or text descriptions who are ready to run the model locally. The open source code allows studying the architecture and adapting it to your own tasks. Having the weights on Hugging Face simplifies downloading the ready-made model.

It will not suit those who lack a CUDA-capable GPU with sufficient memory, nor those looking for a commercial out-of-the-box solution: the cc-by-nc-4.0 weights license restricts use to non-commercial purposes. The total weight size of 50.5 GB is also worth factoring in when planning disk space.

MMAudio is a research project with open source code that provides a ready-made model for synthesizing sound from video or text. Running it requires a CUDA-compatible GPU, Python 3.9+, and PyTorch 2.5.1+. The code is free under MIT, but the weights are only available for non-commercial use.