AIWHYS

Models

K2 Horizon

2 synchronized stories carry this tag.

Open & specialistShipped

vLLM 0.30 adds fast restarts, sparse KV offload and watermarking

vLLM 0.30.0 introduced a persistent per-GPU weight-cache daemon for faster engine restarts, HiSparse host-memory offload for sparse-MLA decode, keyed Gumbel-max watermark generation and detection, and support for model families including DeepSeek V4.1 Flash, K2 Horizon and Cohere Compass. The project reports 762 commits from 315 contributors.

Why it matters: The release targets three production bottlenecks at once: restart time, GPU-memory pressure and provenance signaling. Each changes operational state or request semantics, so adoption needs more than a throughput benchmark.

Read storyOriginal source · github.com
Open & specialistShipped

SGLang 0.5.20 expands model serving and replayable RL sampling

SGLang 0.5.20 adds serving support for GLM-5.3-Flash, Hy4-Preview, Qwen3.8-Flash-Next, K2 Horizon and several autoregressive and diffusion models. Its new sampling masks can return the exact token support and sampled-token log probability for each decode step so trainers can replay top-k or top-p rollouts without reconstructing the sampler.

Why it matters: The release broadens the models one serving stack can host and makes reinforcement-learning rollouts more reproducible. That can simplify training infrastructure, but new kernels, schedulers and model adapters create a large correctness and performance…

Read storyOriginal source · github.com