Transformers 5.19 adds multimodal embeddings and expert parallelism
Hugging Face released Transformers 5.19 with EmbeddingGemma 2 support for shared text, image, audio and video embeddings; expert-parallel token dispatch and Trainer support; per-layer cache configuration; and several cache, watermark and continuous-batching fixes.
Why it matters: The release opens a common multimodal retrieval path and changes distributed MoE and cache internals. Those gains are useful for local and training stacks, but the router-output and continuous-batching changes can break custom integrations or alter…