Technology

expert streaming

1 synchronized story carry this tag.

Transformers previews SSD-streamed MoE experts on a 32GB Mac

A Hugging Face engineering prototype streams quantized mixture-of-experts weights from SSD through a proposed Transformers API, running Qwen3.8-Flash-Next with 121GB of experts on a 32GB M5 Mac and publishing code, conversion artifacts and raw comparisons.

Why it matters: Expert streaming could move very large sparse models onto commodity unified-memory systems, but it trades memory pressure for storage traffic, host round trips, conversion time and model-specific code.

Read storyOriginal source · huggingface.co