Embedding.

A numerical vector representation of text, code, or images that captures semantic meaning — similar items have similar vectors, enabling search, clustering, and retrieval.

An embedding is a fixed-length vector of floating-point numbers that represents the meaning of a piece of text. Two sentences with similar meanings will have embedding vectors that are close together in vector space, even if they use completely different words. This property makes embeddings extraordinarily useful for search, recommendation, and classification tasks.

Embedding models are trained separately from chat models. Popular embedding models include OpenAI's text-embedding-3, Cohere Embed, and open-source options like BGE and E5. They are typically much smaller and faster than generative LLMs, and their outputs are stored in vector databases like Pinecone, Weaviate, or pgvector.

The most common application of embeddings is semantic search within Retrieval-Augmented Generation (RAG) systems. When you ask a question, the system embeds your question, searches a vector database for the most similar document chunks, retrieves them, and passes them to an LLM for final answer generation.