How to evaluate embedding models
The Massive Text EmbeddingEmbeddingA numerical vector representation of text, code, or images that captures semantic meaning — similar items have similar vectors, enabling search, clustering, and retrieval.Learn more → BenchmarkBenchmarkA standardized test or set of tasks used to evaluate and compare the capabilities of different AI models on a common scale.Learn more → (MTEB) is the standard leaderboard for comparing embedding models across 58 tasks and 112 languages. Check mteb.leaderboard.xyz before choosing a model — rankings change as new models are released. Sort by your specific task type (retrieval, clustering, classification) and language.
Benchmark scores matter but are not the only factor. Also consider: vector dimension (higher = more memory), max sequence length (longer text needs higher limit), inferenceInferenceThe process of running a trained AI model to generate outputs — what happens when you send a prompt and receive a response.Learn more → speed (local models vary widely), and whether the model supports instruction-following (some require prepending a task instruction to the input).
OpenAI text-embedding-3
`text-embedding-3-small` (1536 dimensions, $0.02/M tokens) is the best value cloud embedding modelEmbedding ModelA specialized neural network that converts text into high-dimensional vectors, enabling semantic search, clustering, and retrieval applications.Learn more → for most English use cases. It outperforms its predecessor `ada-002` at 5× lower cost. `text-embedding-3-large` (3072 dimensions) is the highest quality OpenAI embedding, worth the cost for applications where retrieval precision is critical.
A unique feature of text-embedding-3 models is dimension reduction: you can request fewer dimensions (e.g. 256) without significant quality loss. Halving the dimensions roughly halves storage and query cost with only ~5% quality reduction — useful for very large indices.
BAAI/bge-m3: the open-source leader
BAAI/bge-m3 is the best open-source multilingual embedding model, supporting 100+ languages with strong English performance that matches or exceeds OpenAI's models on most retrieval benchmarks. It produces 1024-dimensional vectors and supports up to 8192 tokens — ideal for embedding long documents.
BGE-M3 supports three retrieval modes in a single model: dense retrieval (standard cosine similarityCosine SimilarityA metric that measures similarity between two vectors by calculating the cosine of the angle between them, widely used in vector search and retrieval systems.Learn more →), sparse retrieval (like BM25), and multi-vector retrieval (ColBERT-style late interaction). Using all three in a hybrid setup achieves state-of-the-art retrieval quality.
Choosing by use case
English-only, cloud deployment, tight budget: `text-embedding-3-small`. English-only, highest quality: `text-embedding-3-large` or Cohere embed-english-v3.0. Multilingual: `BAAI/bge-m3` (local) or Cohere embed-multilingual-v3.0 (cloud). Privacy-sensitive (no cloud): `BAAI/bge-m3` or `mixedbread-ai/mxbai-embed-large-v1`.
For code search (finding code by natural language description), use a code-specific embedding model: `Salesforce/SFR-Embedding-Code-2B` for highest quality, or `microsoft/codebert-base` for fast local inference. General text embedding models work poorly for cross-modal text-to-code retrieval.