VRAM.

VRAM is the specialized memory found on graphics processing units (GPUs) that serves as the primary bottleneck for running large language models. Unlike system RAM, VRAM is directly accessible by GPU cores at extremely high bandwidth, making it essential for the parallel computations required by neural networks. When serving LLMs, VRAM must simultaneously hold the model's parameters (weights) and the key-value cache that stores attention states for processed tokens, creating a fundamental constraint on both model size and maximum context length.

The VRAM requirement scales predictably with model parameters and context usage. Model weights consume a fixed amount based on parameter count and precision (typically 2 bytes per parameter for 16-bit inference), while the KV cache grows linearly with context length and batch size. Modern consumer GPUs range from 8GB to 24GB of VRAM, while enterprise cards can exceed 80GB. This memory hierarchy determines which models can run locally versus requiring cloud inference, as running out of VRAM forces expensive memory swapping that destroys performance.

VRAM limitations drive many practical decisions in AI deployment, from model selection to optimization strategies. Techniques like quantization reduce memory usage by lowering precision, while methods like offloading move less-critical data to slower system RAM. Many developers must choose between smaller, faster models that fit in available VRAM versus larger, more capable models that require cloud services. Understanding VRAM constraints is crucial for cost-effective AI deployment, as insufficient memory can make otherwise powerful hardware unusable for modern LLMs.