SGLang 0.5.21 adds decision APIs and runtime mode switching
SGLang released signed version 0.5.21 with a low-latency /v1/decisions classifier and scorer API, /v1/score, on-the-fly switching between prefill and decode modes, and a Rust prefix cache enabled by default. The release also expands model, diffusion and AMD MI355X support and publishes NVIDIA, AMD and Intel GPU and CPU images.
Why it matters: Inference operators can route bounded scoring work through a dedicated endpoint and rebalance prefill and decode capacity without restarting the service. A new default cache implementation and a 779-pull-request release also create a large correctness,…