LocalAI v4.10.0: cluster dashboard, credentials file and benchmark

The LocalAI inference tool has released v4.10.0. The authors describe three areas of work: cluster management, loading models from private sources, and backend fixes. Details are in the GitHub release.

What it means

For those running LocalAI on multiple machines, the release provides a cluster operations dashboard instead of a flat list of nodes: aggregate status, VRAM, RAM, CPU and disk usage, a tab of running models with replica placement across workers, and bulk actions drain, resume and remove. Workers now report CPU telemetry on registration and heartbeat. For loading models from private sources, a credentials.yaml file has appeared: it authenticates OCI registries, galleries and direct downloads without scattering tokens across environment variables. The local-ai benchmark command measures end-to-end latency and throughput from the CLI. The ds4 backend received fixes that make it reliable on CUDA: kernels are built for the target architecture, abandoned inference is canceled, generation boundaries are respected, and prefilled reasoning is separated from content. Distributed mode has become more stable thanks to fixes to staging, liveness, virtual models and the control-plane database. Four CVEs were also patched, enable_thinking=false is now honored in all text backends, and Apple M5 no longer crashes on startup. Link to the news feed about AI tools for 3D and video: cpu3d.com/ainews.