Technology

model loading

1 synchronized story carry this tag.

SGLang 0.5.22 widens hardware support and fixes hybrid-state serving

SGLang 0.5.22 adds new model and accelerator paths, a 10x-faster InstantTensor loader claim, Apple Silicon support through PyTorch and multiple speculative-decoding optimizations. It also fixes a correctness issue where hybrid linear models could resume from the wrong recurrent state when structured output and speculative decoding were combined.

Why it matters: The release mixes throughput claims with a concrete correctness repair on a high-risk serving combination. Operators gain wider deployment options, but Python and runtime changes make a controlled upgrade more important than the headline performance numbers.

Read storyOriginal source · github.com