What changed
Qwen released Qwen3-Next-80B-A3B models, activating about three billion parameters per token and combining gated delta-style linear attention with conventional attention for long-context inference.
Why it matters
The architecture targeted the central open-model constraint: getting useful reasoning and long context without paying the memory and compute cost of a dense model on every token.
What to do
Evaluate it for document collections, long agent traces, and high-volume extraction; measure memory use, time to first token, and accuracy across the actual context lengths you need.
Caveat
Architecture efficiency and benchmark gains were Qwen-reported, and sparse or hybrid-attention models may need serving-stack support before their theoretical savings appear in production.