Autoregressive.

Autoregressive generation is the fundamental approach used by most large language models to produce text, where the model generates sequences one token at a time in left-to-right order. Each new token is predicted based on the probability distribution conditioned on all previously generated tokens in the sequence. This sequential dependency creates a natural flow where the model builds meaning incrementally, making it particularly effective for tasks requiring coherent, contextually appropriate text generation.

The autoregressive process works by maintaining a running context of all previously generated tokens and using this history to predict the most likely next token. At each step, the model computes probability distributions over its entire vocabulary, then samples or selects the next token according to various sampling strategies. This creates a causal dependency where future tokens cannot influence past ones, ensuring the generation process remains computationally tractable while preserving the logical flow of language.

While autoregressive generation enables remarkably coherent long-form text, it comes with inherent limitations including slower inference speeds due to sequential processing and potential error propagation where early mistakes compound throughout the sequence. The approach also makes certain tasks like text editing or bidirectional reasoning more challenging compared to non-autoregressive alternatives. Despite these trade-offs, autoregressive generation remains the dominant paradigm because it naturally captures the sequential nature of human language and produces highly coherent outputs.