Encoder.

The encoder is a fundamental component of transformer architectures that transforms raw input sequences into rich, contextual representations. Unlike decoders that generate text sequentially, encoders process entire input sequences simultaneously, creating deep understanding of relationships between all tokens. This bidirectional processing makes encoders particularly powerful for tasks requiring comprehensive text understanding rather than generation, which is why encoder-only models like BERT revolutionized natural language understanding tasks.

Encoders work through a stack of identical layers, each containing multi-head self-attention mechanisms and feed-forward networks. The self-attention allows each token to attend to every other token in the sequence, building contextual representations that capture semantic relationships, syntactic dependencies, and long-range connections. This parallel processing differs fundamentally from the sequential, causal attention used in decoder architectures, enabling encoders to build richer representations of input meaning.

In practice, encoder architectures excel at classification, sentiment analysis, question answering, and other understanding tasks where the model needs to comprehend rather than generate text. While modern large language models primarily use decoder-only architectures for their generative capabilities, encoders remain crucial in specialized applications and hybrid encoder-decoder models. The key trade-off is that encoders sacrifice generative ability for superior understanding, making them ideal when interpretation rather than creation is the primary goal.