What text embeddings are
A text embeddingEmbeddingA numerical vector representation of text, code, or images that captures semantic meaning — similar items have similar vectors, enabling search, clustering, and retrieval.Learn more → is a dense vector — typically 768 to 3072 floating-point numbers — where the values capture the semantic meaning of the text. Texts with similar meaning produce vectors that are close together in this high-dimensional space. This is why embeddings power search, clustering, and similarity detection.
Embeddings are not interpretable — you cannot look at a single number and understand what it means. They only make sense in relation to other embeddings. The distance between two embedding vectors tells you how semantically similar the corresponding texts are.
Generate embeddings with the OpenAI API
Python: `from openai import OpenAI; client = OpenAI(); response = client.embeddings.create(model='text-embedding-3-small', input='The quick brown fox jumps over the lazy dog'); vector = response.data[0].embedding`. The vector is a list of 1536 floats.
For bulk embedding, pass a list of strings in a single call: `client.embeddings.create(model='text-embedding-3-small', input=['sentence 1', 'sentence 2', ...])`. This is much more efficient than one call per text. Batch up to 2048 strings per request. For very large datasets, use the Batch API for async processing at 50% lower cost.
Generate embeddings locally with Sentence Transformers
For privacy or cost reasons, run embedding models locally: `pip install sentence-transformers`. Then: `from sentence_transformers import SentenceTransformer; model = SentenceTransformer('BAAI/bge-m3'); embeddings = model.encode(['sentence 1', 'sentence 2'], batch_size=32, show_progress_bar=True)`.
BAAI/bge-m3 is the leading open-source multilingual embedding modelEmbedding ModelA specialized neural network that converts text into high-dimensional vectors, enabling semantic search, clustering, and retrieval applications.Learn more → (supporting 100+ languages) and matches OpenAI's embedding quality on most benchmarks. For English-only use, `mixedbread-ai/mxbai-embed-large-v1` is a strong alternative. Both run comfortably on CPU; a GPU speeds up bulk encoding 10–20×.
Measure similarity
Cosine similarityCosine SimilarityA metric that measures similarity between two vectors by calculating the cosine of the angle between them, widely used in vector search and retrieval systems.Learn more → is the standard metric for embedding comparison. In NumPy: `import numpy as np; def cosine_similarity(a, b): return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))`. The result is between -1 (opposite) and 1 (identical). For practical search, values above 0.85 are typically highly relevant.
For batch similarity (comparing one query against many stored embeddings), use vectorised operations: `similarities = np.dot(query_vec, stored_vecs.T)`. This computes all similarities in a single matrix multiplication, which is orders of magnitude faster than computing each similarity individually.
Practical applications
Semantic searchSemantic SearchSemantic search retrieves information based on meaning and context using vector embeddings rather than exact keyword matches.Learn more →: embed all your documents at indexing time. At query time, embed the query and find the most similar documents by cosine similarity. This finds relevant documents even when they don't share keywords with the query.
Deduplication: embed all items and cluster by similarity. Items with cosine similarity above 0.95 are near-duplicates. Use this to clean training datasets, deduplicate support tickets, or find similar products.
Classification without training: embed your categories as text descriptions, embed the item to classify, and assign the category with the closest embedding. Zero-shotZero-ShotThe ability of a model to perform a task correctly from just a description, with no examples — demonstrating that it has generalized the concept from training.Learn more → classification that requires no labelled training data.