Turns memories and queries into vectors for recall.
- Parameters: 112M
- Dimensions: 384 (also 256, 128, 64)
- Context: 8,192 tokens
- Languages: 8
- Runs on: CPU or GPU, stock TEI
| English retrieval | Multilingual retrieval | Long conversations | Long inputs, each model at its own limit | Memory retrieval | Serving on CPU | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Model | Overall mean | NanoBEIR | MTEB English v2 retrieval | MIRACL | Belebele, Portuguese | Belebele, Traditional Chinese | BEAM 1M | BEAM 10M | LongEmbed, long documents | LongEmbed, needle and passkey | BEAM 100K, Hindsight-sized chunks | BEAM 1M, Hindsight-sized chunks | LMEB | Query latency ms | Throughput /s |
 hs-embed-1 Hindsight · 112M | 56.8 | 61.3 | 50.8 | 52.0 | 91.6 | 91.5 | 37.5 | 31.4 | 58.6 | 75.1 | 46.5 | 32.7 | 52.3 | 16.5 | 60.1 |
 multilingual-e5-small Microsoft · 118M | 50.3 | 58.2 | 46.5 | 53.8 | 90.6 | 90.9 | 37.2 | 30.1 | 41.0 | 34.5 | 40.7 | 30.4 | 49.8 | 26.3 | 38.6 |
 granite-embedding-97m-multilingual-r2 IBM · 97M | 54.6 | 58.7 | 50.1 | 51.6 | 90.8 | 87.8 | 37.5 | 25.6 | 64.2 | 68.1 | 42.6 | 31.3 | 46.9 | 75.4 | 4.2 |
 bge-small-en-v1.5 BAAI · 33M | 40.9 | 62.6 | 53.9 | 19.6 | 70.0 | 32.9 | 35.9 | 31.1 | 36.2 | 23.9 | 44.1 | 30.6 | 49.5 | 26 | 43.8 |
 snowflake-arctic-embed-s Snowflake · 33M | 40.1 | 62.4 | 54.8 | 22.0 | 78.0 | 9.7 | 32.8 | 25.2 | 43.0 | 32.5 | 42.8 | 27.6 | 50.0 | 26.7 | 43.3 |
e5-small-v2 Microsoft · 33M | 42.3 | 61.0 | 48.5 | 19.3 | 78.4 | 34.2 | 35.0 | 29.9 | 44.2 | 33.8 | 43.4 | 28.6 | 50.9 | 26.3 | 44.6 |
 all-MiniLM-L6-v2 SBERT · 23M | 34.7 | 56.2 | 42.9 | 10.7 | 54.6 | 21.4 | 37.6 | 25.9 | 33.9 | 21.6 | 42.2 | 27.7 | 41.5 | 14.1 | 88 |
Quality vs. speed across 7 models
Top left is best value. Hover a point for details.
Model, as its provider’s logoBest value: no other model is both faster and better
Speed from the Query latency column, on CPU.
Benchmarks
Scores are shown 0 to 100, higher is better; speed and input size as measured, with lower latency better. A shaded, outlined cell is the best in its column: blue when it is ours. Overall is the mean over the score columns.
- NanoBEIRnDCG@10, 13 tasks
- MTEB English v2 retrievalnDCG@10, 10 tasks
- MIRACLnDCG@10, 7 languages
- Belebele, PortuguesenDCG@10
- Belebele, Traditional ChinesenDCG@10
- BEAM 1MEvidence turns in the top 10 (%), 625 questions
- BEAM 10MEvidence turns in the top 10 (%), 176 questions
- LongEmbed, long documentsnDCG@10, NarrativeQA, QMSum, SummScreenFD, 2WikiMultihopQA
- LongEmbed, needle and passkeynDCG@1, 256 to 32,768 tokens
- BEAM 100K, Hindsight-sized chunksEvidence chunks in the top 10 (%), 3,000-character chunks
- BEAM 1M, Hindsight-sized chunksEvidence chunks in the top 10 (%), 3,000-character chunks
- LMEBnDCG@10, 17 datasets
- Query latencyMedian ms per query, 2 CPU threads
- ThroughputMemories embedded per second, 2 CPU threads