Transformer.

The neural network architecture that underpins virtually all modern LLMs, introduced in 2017, built around self-attention mechanisms that process entire sequences in parallel.

The Transformer architecture was introduced in the 2017 paper 'Attention Is All You Need' by Vaswani et al. at Google Brain. It replaced sequential recurrent networks (RNNs and LSTMs) with self-attention, which processes all tokens in a sequence simultaneously and can directly model long-range dependencies. This parallelism made transformers dramatically more efficient to train on modern hardware.

A standard transformer consists of stacked encoder and/or decoder blocks. Each block contains a multi-head self-attention layer, a feed-forward network, residual connections, and layer normalization. GPT-style models use only the decoder component (they predict the next token); BERT-style models use only the encoder. T5 and other models use both.

The transformer's impact cannot be overstated. It is the architecture behind GPT, Claude, Gemini, Llama, DALL-E, Stable Diffusion, AlphaFold, and most other frontier AI systems. Its combination of scalability, expressiveness, and compatibility with GPU parallelism made it the dominant deep learning building block of the 2020s.