CPU3DAI tools for 3D, video and audio

ByteDance-Seed releases Cola2 language model on HuggingFace

The model ByteDance-Seed/Cola2 with the text-generation task has been published on HuggingFace. It is a continuous-latent language model: it compresses text into continuous latent representations, generates them in blocks via a Diffusion Transformer, and then expands them back into discrete tokens. The checkpoint was trained on 346.03 billion tokens, and the code has been released as open source.

What it means

The model contains 8.8 billion parameters; the weights in the repository take up 32.7 GB across eight sharded safetensors files, the largest of which is 4.7 GB. It runs via transformers, and the architecture is a Diffusion Transformer. There are no quantized builds on the platform yet, so running it locally will require a GPU with enough VRAM headroom for the full model weight. The author did not specify a license. For those following local deployment of generative models, Cola2 is interesting primarily as a research work: latent generation is stochastic, and results on your own hardware may differ from those reported in the paper. You can keep track of quantized builds and related news in the CPU3D news feed.
ByteDance-Seed Cola2 — continuous-latent language model Text compression into latents, diffusion generation, decoding into tokens Input text discrete tokens Encoder compression into latents Diffusion Transformer block-wise latent generation stochastic generation Decoder into tokens Output text Model specs 8.8B parameters full model weight 32.7 GB weights in the repo 8 sharded safetensors 346.03B tokens trained training set 4.7 GB largest file safetensors shard No quantized builds — you need a GPU with enough VRAM for the full weight
How the method works. The diagram was drawn based on this news note.

See also