Sentence Transformers v6.0 Adds MultiVectorEncoder for ColBERT Search

Summary

Published on August 18, 2026, by Tom Aarsen, Antoine Chaffin and Raphael Sourty, Sentence Transformers v6.0 adds MultiVectorEncoder, its fourth model type, for ColBERT-style late-interaction retrieval across text, page images, audio and video. Rather than compressing a document into one 384, 768 or 1024 number vector, it retains a classically 128-dimensional vector per token and applies MaxSim, summing each query token’s best document-token cosine match. This preserves exact identifiers, clauses, synonyms and multiple requirements while documents remain precomputable, unlike cross-encoders. PyLate and Stanford-NLP ColBERT checkpoints load directly. ColPali-family models support OCR-free page search but need repository configuration, much of it awaiting merger. Version 6.0 requires transformers v5.x, torch 2.2+ and huggingface-hub v1.x and includes breaking changes.

On 4,874 Natural Questions passages, lightonai/LateOn produced 608,414 token vectors, averaging 124.8 per passage: 311.5 MB in float32, about 42 times all-MiniLM-L6-v2’s 7.5 MB and above gte-modernbert-base’s 15.0 MB. fast-plaid reduced that to 92 MB, near Qwen3-Embedding-8B’s roughly 80 MB, indexed it in 5 seconds and searched in 11 ms on an RTX 3090. Exhaustive encoding took 20 seconds and queries about 120 ms, while Qdrant ingested in 26.3 seconds and searched in 18 ms. mLateOn scored 77.92 on MLDR versus mDenseOn’s 51.59. Token Pooling, dense retrieval followed by MaxSim reranking of 50 candidates, and support from Qdrant 1.10, Weaviate 1.29, Vespa, LanceDB 0.15.0, VectorChord and Milvus 2.6.4 address scale. LateOn’s 300-token cap reduced a 662-token passage to 273 vectors, while ColBERTv2 fixes queries at 32 tokens and documents at 180.