The AI glossary, in plain language.
Every term you hit when reading about AI at work, defined without the jargon that usually comes with it. 182 entries. They also appear as hover definitions throughout the site, so you rarely have to come here at all.
A
- Activation FunctionMathematical functions like ReLU, GELU, or SwiGLU that introduce nonlinearity into neural networks, enabling them to learn complex patterns beyond linear relationships.
- AgentAn LLM-powered system that can take actions, use tools, and pursue multi-step goals autonomously without human input at each step.
- Agent MemorySystems that enable AI agents to store, retrieve, and use information across multiple interactions or sessions, extending beyond the model's context window.
- Agentic CodingAI-assisted software development where the AI operates autonomously across multiple steps — reading files, executing code, observing results, and iterating — with minimal human intervention per action.
- AIMEAIME (American Invitational Mathematics Examination) problems are challenging high-school competition math questions used to evaluate advanced mathematical reasoning capabilities in AI models.
- AlignmentThe process of ensuring AI models behave according to human values and intentions through training techniques, evaluation methods, and safety measures.
- ARCThe AI2 Reasoning Challenge (ARC) is a benchmark dataset of grade-school science questions designed to test AI models' reasoning abilities.
- Attention MechanismA technique that allows each token in a sequence to 'pay attention' to all other tokens, enabling the model to understand context and relationships across long distances.
- AutoregressiveA text generation approach where models produce sequences one token at a time, with each new token conditioned on all previously generated tokens.
B
- BackpropagationThe algorithm that computes loss gradients for every parameter by applying the chain rule backward through the network.
- Batch SizeThe number of training examples processed together in a single forward and backward pass before updating model parameters.
- Beam SearchA decoding algorithm that maintains the top-k most probable partial sequences at each generation step to find higher-quality complete outputs.
- BenchmarkA standardized test or set of tasks used to evaluate and compare the capabilities of different AI models on a common scale.
- BLEUBLEU is a precision-based metric that evaluates machine translation quality by comparing n-gram overlap between generated text and reference translations.
- Byte-Pair EncodingA subword tokenization algorithm that iteratively merges the most frequent pairs of bytes or characters to build a vocabulary that balances size and sequence length.
C
- Catastrophic ForgettingThe phenomenon where neural networks lose previously learned knowledge when trained on new tasks or data, requiring careful strategies to preserve existing capabilities.
- Chain of ThoughtA prompting technique that encourages an LLM to reason step by step before giving a final answer, dramatically improving performance on complex tasks.
- Chatbot ArenaA public platform where users anonymously compare language models in head-to-head battles, with results aggregated into Elo ratings to create crowdsourced leaderboards.
- CheckpointA saved snapshot of model weights and training state that allows resuming training, evaluation, or deployment from a specific point.
- ChunkingThe process of splitting documents into smaller passages for embedding and retrieval in RAG systems, where chunk size and overlap affect retrieval quality.
- Code CompletionAn AI feature that predicts and suggests the next tokens of code as a developer types, ranging from single-token suggestions to entire function implementations.
- Codebase IndexingThe process of analyzing and converting entire code repositories into searchable embeddings or symbol maps that enable AI coding tools to quickly find and retrieve relevant code context.
- Constitutional AIAnthropic's technique for training safe AI models using a written set of principles (a 'constitution') that the model uses to evaluate and revise its own outputs.
- Context WindowThe maximum amount of text (measured in tokens) that a model can read and reason over in a single interaction, including your prompt, any documents, and previous conversation history.
- Context-Aware CompletionAI code suggestions that take into account the full project structure, imported libraries, related files, and coding conventions — not just the current file — to generate more accurate and idiomatic completions.
- Continued PretrainingFurther training of an existing base model on new or domain-specific data to extend its knowledge without starting from scratch.
- Continuous BatchingA serving optimization technique that dynamically adds and removes requests from GPU batches during execution to maximize throughput and hardware utilization.
- Cosine SimilarityA metric that measures similarity between two vectors by calculating the cosine of the angle between them, widely used in vector search and retrieval systems.
- Cross-AttentionAn attention mechanism where queries come from one sequence while keys and values come from another, enabling models to relate information across different inputs or modalities.
- Cross-EntropyA loss function that measures the difference between predicted token probabilities and actual tokens, serving as the primary training objective for language models.
D
- Data ContaminationWhen test questions from evaluation benchmarks accidentally appear in training data, artificially inflating model performance scores.
- DecoderA transformer component that generates text tokens sequentially using causal attention, where each token can only attend to previous tokens in the sequence.
- Diffusion ModelA generative AI model that creates data by learning to reverse a noise-adding process, iteratively denoising random inputs to produce high-quality outputs.
- Direct Preference OptimizationA training method that aligns language models with human preferences directly from preference pairs, bypassing the need for separate reward models or reinforcement learning.
- DistillationA technique where a smaller student model is trained to replicate the behavior of a larger teacher model, achieving similar performance with reduced computational requirements.
- DropoutA regularization technique that randomly sets neural network activations to zero during training to prevent overfitting and improve generalization.
E
- Elo RatingA rating system borrowed from chess that ranks AI models based on head-to-head comparisons from human preference battles.
- EmbeddingA numerical vector representation of text, code, or images that captures semantic meaning — similar items have similar vectors, enabling search, clustering, and retrieval.
- Embedding ModelA specialized neural network that converts text into high-dimensional vectors, enabling semantic search, clustering, and retrieval applications.
- Emergent AbilitiesCapabilities that appear suddenly in large language models only after reaching a certain scale threshold, often unpredictable from smaller model performance.
- EncoderThe transformer component that processes input sequences into contextual representations, forming the foundation of understanding-focused models like BERT.
- EpochOne complete pass through an entire training dataset during machine learning model training, representing a fundamental unit of the training process.
F
- F1 ScoreThe harmonic mean of precision and recall, providing a single balanced metric that equally weighs both false positives and false negatives in classification tasks.
- Feed-Forward NetworkThe position-wise multilayer perceptron within each transformer block that processes tokens individually after attention, containing most of the model's parameters.
- Few-ShotA prompting technique where you include a small number of input-output examples in the prompt to demonstrate the desired task format and behavior to the model.
- Fine-tuningThe process of further training a pre-trained model on a smaller, task-specific dataset to specialize its behavior for a particular use case.
- FlashAttentionAn IO-aware attention algorithm that tiles computation in fast SRAM memory to reduce data movement, enabling efficient training and inference on long sequences.
- FLOPsFloating-point operations per second, a measure of computational throughput used to quantify the processing power required for training and running AI models.
- Foundation ModelA large AI model trained on broad data at massive scale that can be adapted to a wide variety of downstream tasks, forming the 'foundation' for specialized applications.
- Frequency PenaltyA sampling parameter that reduces the probability of selecting tokens based on how frequently they've already appeared in the generated text.
- Frontier ModelThe most advanced and capable AI models at the cutting edge of the field, typically developed by well-funded research labs with massive computational resources.
- Function CallingA capability that allows language models to return structured calls to developer-defined functions or tools, enabling AI systems to interact with external APIs, databases, and services.
G
- GGUFA file format for distributing quantized language models optimized for efficient local inference, popularized by llama.cpp.
- GPQAA graduate-level, 'Google-proof' multiple-choice benchmark of hard science questions designed to resist easy lookup and test deep reasoning.
- GPTGenerative Pre-trained Transformer — the model architecture and family name behind OpenAI's most famous models, from GPT-2 to GPT-5.
- Gradient AccumulationA training technique that accumulates gradients across multiple mini-batches before applying parameter updates, effectively simulating larger batch sizes within memory constraints.
- Gradient DescentAn optimization algorithm that iteratively adjusts model parameters by moving in the direction opposite to the gradient of the loss function.
- Greedy DecodingA text generation strategy that always selects the token with the highest probability at each step, producing deterministic but potentially repetitive output.
- Ground TruthThe reference correct answer used to score model outputs during evaluation, serving as the objective standard against which AI performance is measured.
- GroundingThe process of connecting an LLM's outputs to verified external information sources to improve accuracy and reduce hallucinations.
- Grouped-Query AttentionAn attention optimization technique where multiple attention heads share key and value projections in groups, reducing memory usage and speeding up inference while maintaining model quality.
- GSM8KA benchmark dataset of 8,500 grade-school math word problems that tests language models' ability to perform multi-step arithmetic reasoning.
- GuardrailsInput/output checks and policies layered around AI models to prevent harmful, inappropriate, or off-topic responses through filtering and validation.
H
- HallucinationWhen an AI model generates text that is factually incorrect, fabricated, or unsupported by its context, stated with confident fluency as if it were true.
- HellaSwagA commonsense reasoning benchmark where AI models must select the most plausible continuation from four options for everyday scenarios.
- Human in the LoopA design pattern where humans review, approve, or intervene in AI decisions before they are executed, particularly for high-stakes or irreversible actions.
- HumanEvalA benchmark dataset of 164 Python programming problems used to evaluate code generation capabilities of language models through unit test pass rates.
I
- In-Context LearningThe ability of an LLM to learn from examples provided directly in the prompt, without any weight updates or training — just from seeing patterns in context.
- InferenceThe process of running a trained AI model to generate outputs — what happens when you send a prompt and receive a response.
- Inline EditAn AI editing mode where you select code in your editor, describe a change in natural language, and the AI applies the change in place — showing a diff for review before accepting.
- Instruct ModelA base language model that has been fine-tuned to follow instructions from users, as opposed to just completing text in the style of its training data.
- InterpretabilityResearch into understanding how AI models work internally, including mechanistic interpretability that maps specific behaviors to neural components.
J
K
L
- LatencyThe delay between sending a request to an LLM and receiving the first token of the response, often measured as Time to First Token (TTFT).
- Layer NormalizationA normalization technique that standardizes activations across features within each layer to stabilize training and improve convergence speed.
- Learning RateThe step size that determines how much a neural network's parameters change during each optimization update, typically starting small, warming up, then decaying.
- LLMLarge Language Model — a neural network trained on vast amounts of text data that can understand and generate human-quality language across a wide range of tasks.
- LLM-as-a-JudgeUsing a powerful language model to evaluate, score, or rank outputs from other AI models, offering scalable assessment but introducing systematic biases.
- Logit BiasAn API parameter that adjusts the probability of specific tokens during generation by adding or subtracting values from their logits.
- LogitsRaw, unnormalized numerical scores that language models output for each possible next token before applying softmax to convert them into probabilities.
- Long ContextThe ability of AI models to process extremely large inputs spanning tens of thousands to millions of tokens, enabling comprehensive analysis of entire documents, codebases, or datasets in a single context.
- LoRALow-Rank Adaptation (LoRA) is a parameter-efficient fine-tuning technique that trains small weight matrices while freezing the base model.
- Loss FunctionA mathematical function that quantifies how far a model's predictions are from the correct answers, guiding optimization during training.
M
- Max TokensThe maximum number of tokens an LLM can generate in a single response, separate from the context window that determines input capacity.
- Mixed PrecisionA training technique that uses lower-precision floating-point formats (fp16/bf16) instead of fp32 to reduce memory usage and increase speed while maintaining model accuracy.
- Mixture of ExpertsA neural network architecture where only a subset of model parameters are activated for each input token, enabling very large models that are surprisingly efficient to run.
- MMLUA comprehensive benchmark evaluating language models across 57 academic subjects from elementary to professional level using multiple-choice questions.
- Model BiasSystematic skews in AI model outputs that reflect unfair patterns, stereotypes, or imbalances present in training data, creating fairness and safety concerns.
- Model CardA standardized document that transparently describes a machine learning model's intended use, training data, capabilities, limitations, and potential risks.
- Model CollapseThe degradation in quality and diversity that occurs when AI models are trained on data generated by other AI models, creating a feedback loop of declining performance.
- Model Context ProtocolAn open protocol that standardizes how LLM applications connect to external tools and data sources through intermediary servers.
- Model RoutingModel routing automatically directs each request to the most suitable AI model based on factors like cost, capability, latency, or task requirements.
- ModerationAutomated classification systems that scan AI model inputs and outputs to detect and filter unsafe, harmful, or policy-violating content before it reaches users.
- MT-BenchA multi-turn conversation benchmark that evaluates LLMs using GPT-4 as a judge to score responses across diverse, challenging dialogue scenarios.
- Multi-AgentSystems where multiple AI agents with distinct roles collaborate, debate, or critique each other to solve complex tasks more effectively than single agents.
- Multi-Head AttentionA mechanism that runs multiple attention operations in parallel, each learning different aspects of relationships between tokens.
- MultimodalCapable of processing and generating multiple types of data — such as text, images, audio, and video — within a single model.
O
- One-ShotA prompting technique where exactly one example is provided to demonstrate the desired task format before presenting the actual query.
- Open-WeightA model whose trained weights are publicly available for download, allowing anyone to run, fine-tune, or build on top of it — distinct from fully open-source (which also includes training code and data).
- OptimizerAn algorithm that uses gradients to update neural network weights during training, with techniques like momentum and adaptive learning rates.
- OverfittingWhen a machine learning model memorizes training data too closely, performing well on training examples but failing to generalize to new, unseen data.
P
- PagedAttentionA memory management technique from vLLM that allocates attention cache in pages like an operating system, reducing memory waste and enabling higher concurrency.
- ParametersThe numerical weights inside a neural network that are learned during training — the 'knowledge' of the model, measured in billions for modern LLMs.
- Pass@kPass@k measures the probability that at least one of k generated code solutions passes all test cases, evaluating coding model reliability.
- PEFTParameter-Efficient Fine-Tuning (PEFT) adapts large language models by training only a small subset of parameters, dramatically reducing computational costs while maintaining performance.
- PerplexityA statistical measure of how well a language model predicts text — lower perplexity means the model finds the text less surprising and is a better predictor.
- PIIPersonally Identifiable Information (PII) refers to data that can identify specific individuals, requiring careful handling when using AI models to prevent privacy breaches.
- PlanningAn agent's decomposition of a goal into an ordered set of steps or subgoals before or while acting to achieve complex objectives systematically.
- Positional EncodingA technique that adds position information to tokens in transformer models, enabling them to understand word order despite attention being inherently position-agnostic.
- Post-trainingThe phase after pretraining where base language models are refined through instruction tuning, RLHF, and safety measures to become helpful assistants.
- PrefillThe initial inference phase where an LLM processes the entire input prompt in parallel to populate the KV cache before beginning sequential token generation.
- Presence PenaltyA sampling parameter that reduces the probability of tokens that have already appeared in the text, encouraging models to introduce new topics and vocabulary.
- PretrainingThe initial large-scale training phase where language models learn patterns from vast text datasets through next-token prediction.
- PromptThe input text sent to a language model — the question, instruction, or context that triggers a response.
- Prompt CachingA technique that stores and reuses computed results from repeated prompt prefixes across API requests, reducing both response latency and computational costs.
- Prompt EngineeringThe practice of crafting inputs, instructions, examples, and formatting to guide language models toward producing better, more accurate outputs.
- Prompt InjectionA security attack where malicious instructions hidden in untrusted input manipulate an LLM to ignore its original instructions and execute harmful commands.
- Prompt TemplateA reusable, parameterized prompt structure with placeholders that get filled with specific values at runtime to create consistent LLM interactions.
- Proprietary ModelA closed AI model where the weights and architecture remain private, accessible only through controlled APIs rather than direct downloads.
R
- RAGRetrieval-Augmented Generation — a technique that enhances LLM responses by first retrieving relevant documents from an external knowledge base and including them in the prompt.
- Rate LimitRestrictions on the number of API requests or tokens that can be processed within a specific time period to manage server capacity and costs.
- ReActA prompting pattern that alternates between reasoning steps and tool-use actions, enabling models to think through problems, take actions, observe results, and iterate.
- Reasoning ModelA class of LLMs trained specifically to 'think' through problems step by step using extended chain-of-thought reasoning before producing a final answer, excelling at math, coding, and complex logic.
- Red-TeamingA systematic approach to adversarially testing AI models by attempting to elicit harmful, unsafe, or policy-violating behaviors before deployment.
- RegularizationTechniques that prevent neural networks from overfitting by adding constraints or noise during training to improve generalization to new data.
- Repetition PenaltyA decoding parameter that reduces the probability of previously generated tokens to prevent repetitive text and infinite loops during generation.
- RerankingA second-stage model that reorders retrieved candidates by relevance to improve the quality and precision of RAG system outputs.
- Residual ConnectionSkip connections that add a layer's input directly to its output, enabling gradient flow through very deep neural networks.
- Reward HackingWhen an AI model exploits flaws or loopholes in its reward signal to achieve high scores without actually performing the intended task correctly.
- Reward ModelA model trained on human preference data that scores AI outputs, enabling reinforcement learning systems to optimize for human-aligned responses.
- RLAIFReinforcement Learning from AI Feedback uses AI models as judges to generate preference signals for training, replacing human evaluators in the alignment process.
- RLHFReinforcement Learning from Human Feedback — a training technique that uses human preferences to teach AI models to be helpful, honest, and harmless.
- Rotary Position EmbeddingRotary Position Embedding (RoPE) encodes positional information by rotating query and key vectors, enabling attention to depend on relative distances between tokens.
- ROUGEROUGE is a recall-oriented evaluation metric that measures n-gram overlap between generated summaries and reference texts to assess summarization quality.
S
- SamplingThe method used to select each next token from a probability distribution during text generation, controlling the randomness and creativity of model outputs.
- Scaling LawsEmpirical power-law relationships showing how model performance improves predictably with increases in model size, training data, and compute resources.
- Self-ConsistencyA technique that generates multiple chain-of-thought reasoning paths for the same problem and selects the most common answer to improve accuracy.
- Self-Supervised LearningA machine learning approach where models learn from unlabeled data by predicting missing parts of the input, such as the next word in a sentence.
- Semantic SearchSemantic search retrieves information based on meaning and context using vector embeddings rather than exact keyword matches.
- Sliding-Window AttentionA technique that limits each token's attention to a fixed local window of nearby tokens, reducing computational cost for long sequences.
- Small Language ModelA compact language model with fewer parameters designed to run efficiently on consumer devices or at lower cost while maintaining competitive performance.
- SoftmaxA mathematical function that converts a vector of raw logits into a probability distribution, with temperature controlling the sharpness of the output.
- Special TokensReserved tokens like BOS, EOS, padding, and chat-role markers that structure input sequences and signal boundaries to language models.
- Speculative DecodingA technique where a small draft model proposes multiple tokens that a larger model verifies in parallel, maintaining identical output while reducing generation latency.
- Stop SequenceA predefined string that signals an LLM to halt text generation when encountered, used to control output boundaries and structure responses.
- StreamingStreaming returns a model's tokens incrementally as they're generated rather than waiting for the complete response to finish.
- Structured OutputConstraining a language model to produce output that conforms to a predefined format or schema, such as valid JSON, rather than free-form text.
- Supervised Fine-TuningSupervised Fine-Tuning (SFT) is the process of training a pre-trained base model on curated instruction-response pairs to teach it specific tasks before preference alignment.
- SWE-BenchA benchmark that measures an AI agent's ability to resolve real GitHub issues from open-source repositories — a rigorous test of practical software engineering capability.
- SycophancyA model's tendency to prioritize user agreement over accuracy, telling people what they want to hear rather than providing truthful responses.
- Synthetic DataModel- or program-generated training data used to augment or replace human-collected data, central to modern AI training and post-training processes.
- System PromptA special instruction given to a language model before the user conversation begins, establishing the model's persona, capabilities, constraints, and context.
T
- TemperatureA parameter controlling the randomness of model outputs — lower values produce more focused, deterministic responses; higher values produce more creative, varied text.
- Tensor ParallelismA parallelization technique that splits individual weight matrices across multiple GPUs, enabling deployment of models too large for a single device.
- Test-Time ComputeUsing additional computation during inference to improve model outputs through techniques like extended reasoning, multiple sampling, or search algorithms.
- ThroughputThe number of tokens or requests an AI system can process per unit time, measuring overall capacity rather than per-request speed.
- Time to First TokenTime to First Token (TTFT) measures the latency from sending a request to receiving the first streamed token, primarily determined by the prefill phase.
- TokenThe basic unit of text that an LLM processes — roughly corresponding to a word or word-piece. Models read input and produce output in tokens, which is also how API usage is measured and billed.
- TokenizationThe process of breaking raw text into discrete units (tokens) that language models can process, typically using subword algorithms like BPE.
- Tokens per SecondTokens per second measures how quickly an LLM generates text during inference, serving as the primary throughput metric for comparing model speed.
- Tool UseTool use enables language models to call external functions, APIs, or execute code beyond text generation, forming the foundation for AI agents.
- Top-KA sampling parameter that restricts token selection to the k most probable next tokens, discarding the long tail of unlikely options.
- Top-PAlso called nucleus sampling — a method that selects from the smallest set of tokens whose combined probability exceeds a threshold p, adapting dynamically to the model's confidence at each step.
- Transfer LearningA machine learning technique where a model pretrained on large, general datasets is adapted to specific tasks using minimal task-specific data.
- TransformerThe neural network architecture that underpins virtually all modern LLMs, introduced in 2017, built around self-attention mechanisms that process entire sequences in parallel.
- Tree of ThoughtsA prompting strategy that explores multiple branching reasoning paths simultaneously, evaluating different approaches before selecting the best solution.
- TruthfulQAA benchmark that evaluates whether language models can avoid repeating common human misconceptions and provide truthful answers to questions.
V
- Vector DatabaseA database optimized for storing and searching embedding vectors, enabling fast semantic similarity search across millions of text chunks or documents.
- Vibe CodingA style of software development where a developer describes intent in natural language and an AI tool writes the code — shifting the developer's role from typing code to directing an AI.
- Vision-Language ModelA multimodal AI model that processes both images and text to understand visual content and answer questions about what it sees.
- VocabularyThe fixed set of tokens (words, subwords, characters) that a language model can recognize and generate, typically ranging from thousands to millions of unique elements.
- VRAMVideo Random Access Memory (VRAM) is the dedicated memory on graphics cards that stores model weights and the KV cache during LLM inference.
W
- WatermarkingA technique for embedding detectable statistical patterns in AI-generated text to identify content produced by language models.
- WebContainersA browser-native runtime developed by StackBlitz that executes Node.js and npm packages directly in the browser via WebAssembly, enabling full-stack web development without a server.
- Weight DecayA regularization technique that gradually shrinks model weights toward zero during training to prevent overfitting and improve generalization.
- Win RateThe percentage of head-to-head comparisons where one AI model is judged superior to another by human evaluators or automated systems.