Models

Cohere Compass

1 synchronized story carry this tag.

vLLM 0.30 adds fast restarts, sparse KV offload and watermarking

vLLM 0.30.0 introduced a persistent per-GPU weight-cache daemon for faster engine restarts, HiSparse host-memory offload for sparse-MLA decode, keyed Gumbel-max watermark generation and detection, and support for model families including DeepSeek V4.1 Flash, K2 Horizon and Cohere Compass. The project reports 762 commits from 315 contributors.

Why it matters: The release targets three production bottlenecks at once: restart time, GPU-memory pressure and provenance signaling. Each changes operational state or request semantics, so adoption needs more than a throughput benchmark.

Read storyOriginal source · github.com