Sentence Transformers v6.0 Adds MultiVectorEncoder Training for ColBERT Retrieval
Sentence Transformers v6.0 adds MultiVectorEncoder training, with a medical model reaching 0.9139 NDCG@10 after 14.5 hours on one RTX 3090 consumer GPU.
Summary
Sentence Transformers v6.0 introduces MultiVectorEncoder, its fourth model type, with end-to-end training for ColBERT-style late-interaction retrieval. Instead of compressing each text into one vector, it retains one 128-dimensional vector per token and uses MaxSim scoring, preserving fine-grained matches but enlarging indexes. Tom Aarsen trained multi-vector-encoder/mLateOn-medical from lightonai/mLateOn-unsupervised using 1 million pairs from the 4.4 million-pair MIRIAD medical dataset, whose passages average 941 tokens. The run used full documents, an 8,192-token context, punctuation filtering, CachedMultiVectorMultipleNegativesRankingLoss, batch size 128 and a 1e-4 learning rate. It finished in 14.5 hours on one RTX 3090 using 17.5 GB peak VRAM, while punctuation filtering cut index size 9.6%. Training on 100,000 pairs took 75 minutes and finished within 0.012 NDCG@10 of the million-pair model.
Across more than 50 configurations, 1,000 held-out questions and 200,000 passages, mLateOn-medical scored 0.9139 NDCG@10. It beat lightonai/mLateOn at 0.8520, lightonai/GTE-ModernColBERT-v1 at 0.8502, Qwen/Qwen3-Embedding-4B at 0.7817, voyageai/voyage-4-nano at 0.7563, BM25 at 0.7501 and naver/splade-v3 at 0.6853. Rank-one accuracy rose from 75.8% for the strongest zero-shot model to 84.9%, cutting errors by more than one-third. Pre-supervised checkpoints adapted best: mLateOn-unsupervised improved from 0.9087 to 0.9398 after 25,000 pairs, while finished checkpoints barely improved or regressed. Document truncation previously cost up to 0.24 NDCG@10 on long medical passages, and training at 512 tokens sacrificed about 0.015 for roughly twice the speed.
Positives
- mLateOn-medical reached 0.9139 NDCG@10, beating more than 50 dense, sparse, lexical and multi-vector retrieval configurations.
- 84.9% rank-one accuracy reduced first-result errors by more than one-third versus the strongest zero-shot model’s 75.8%.
- 100,000 training pairs required 75 minutes and came within 0.012 NDCG@10 of the full million-pair run.
- Punctuation filtering reduced the document index by 9.6% while delivering a modest retrieval-quality improvement.
- mLateOn-unsupervised rose from 0.9087 to 0.9398 NDCG@10 after only 25,000 medical question-passage pairs.
Risks & concerns
- Token-level embeddings require larger indexes than dense models that store one vector per document.
- Document caps of 180 to 512 tokens can silently discard content, costing up to 0.24 NDCG@10 on 941-token medical passages.
- Training at 512 tokens doubled speed but lost about 0.015 NDCG@10, with additional data failing to erase the deficit.
- Finished retrieval checkpoints barely improved or regressed during domain adaptation, making the intuitive starting point the weakest option.
- The full million-pair run still required 14.5 hours and 17.5 GB peak VRAM on an RTX 3090.