XTTS: audio generator — what it does and what you need to run it

XTTS is an open source speech generator from Coqui. It solves the text-to-speech task: converting text into spoken speech. The project is developed in a GitHub repository and is distributed as a Python library.

What it does

The authors describe the project as a set of deep learning tools for speech synthesis, which, in their words, has been validated in research and in production environments. The repository tags mention multi-speaker-tts, voice-cloning, and voice-conversion — that is, working with multiple voices, voice cloning, and voice conversion. The architectures glow-tts, hifigan, melgan, tacotron, and the speaker-encoder and vocoder components are also mentioned.

The XTTS-v2 model is hosted on Hugging Face. The total size of the model weights files is 1.9 GB, with the largest file taking up 1.7 GB. The weights license is listed as "other," meaning non-standard; the authors do not disclose specific terms in the fact sheet.

What you need to run it

The code is written in Python. The code license is MPL-2.0. The latest release is v0.22.0, and the last code change was on August 16, 2024. The repository was created on May 20, 2020.

As described by the author, the platform is cuda. The description references a CUDA installation guide for Windows written by user GuyPaddock. The authors do not specify memory requirements or specific CUDA versions.

Who it suits

The project suits those looking for an open tool for speech synthesis in Python and willing to handle installation and setup on their own. The presence of voice-cloning and voice-conversion tags may be of interest to those working with multiple voices or voice transfer tasks.

It does not suit those expecting a ready-made application with an interface: this is a library and a model, not a turnkey product. It is also worth noting that the weights license is non-standard — it needs to be reviewed separately before commercial use. The authors do not specify memory requirements, so assessing the minimum configuration in advance is difficult.

XTTS is an open research tool for speech synthesis that has been developed since 2020 and continues to receive updates. It provides access to the model and code but requires technical expertise and attention to the weights license terms.