Audio8-ASR-Infinite: Streaming Speech Recognition with Constant Memory Usage

On September 21, 2026, the model Edge0/Audio8-ASR-Infinite was published on HuggingFace with the text-generation task. The author describes it as native streaming speech recognition designed for minimal latency: the model makes decisions 12.5 times per second, and a sliding KV cache keeps memory consumption and latency at a constant level even during continuous 24/7 operation. Chinese and English are supported, and there is a semantic VAD that distinguishes thinking pauses and stuttering from a real end of utterance.

What it means

The model contains 4.1 billion parameters, the weights take up 7.6 GB in a single model.safetensors file. The license is Apache 2.0, meaning the model can be used and modified, including in commercial projects, with attribution preserved. It runs via the transformers library, but for streaming mode with infinite length the author recommends a customized vLLM build. There are no quantized builds on the platform yet, so running it locally will require a GPU capable of holding the full 7.6 GB of weights plus the working context. For those looking for speech recognition tools that run on your own hardware, the model is interesting primarily for its constant memory consumption: the sliding KV cache keeps usage from growing over time, which is rarely seen in streaming ASR models. You can follow the appearance of quantized versions and new models in this area in the CPU3D news feed.