AIWHYS

Technology

vector quantization

2 synchronized stories carry this tag.

Frontier modelsShipped

Cohere ships Embed 5 with shared Pro and Fast vectors

Cohere released Embed 5 Pro and Fast for text, image and fused retrieval across more than 100 languages and contexts up to 128K tokens. Both models share an embedding space, allowing teams to index with Pro and query with Fast without rebuilding the index, and support Matryoshka dimensions plus float, int8 and binary outputs.

Why it matters: A compatible quality and latency tier can reduce query-path cost without forcing a second corpus index. The multimodal and compressed-output options also make storage, retrieval quality and private-serving requirements explicit engineering choices rather…

Read storyOriginal source · cohere.com
InfrastructurePreprint

TurboQuant targets extreme KV-cache and vector compression

Google presented TurboQuant, PolarQuant and Quantized Johnson–Lindenstrauss methods for compressing high-dimensional vectors with low overhead. The work targets KV caches and vector search, with reported tests on Gemma and Mistral showing large memory and attention-compute gains at very low bit widths.

Why it matters: KV-cache memory can cap context length and concurrency before model weights do, so cache compression is becoming a first-class serving decision.

Read storyOriginal source · research.google