Multimodal.

A multimodal AI model can understand and reason across different modalities simultaneously. Rather than having separate models for vision and language, multimodal models like GPT-4o, Gemini, and Claude 3.5 Sonnet can analyze an image while discussing it in text, understand audio alongside a transcript, or generate code based on a screenshot.

Most current multimodal LLMs are primarily text models with additional encoder components (like CLIP for images) that project other modalities into the model's text embedding space. True native multimodality — where the model processes all modalities with the same architecture from training — is an active research frontier, with Google's Gemini and Meta's Llama 4 being early examples.

Multimodal capabilities unlock a wide range of applications: analyzing medical imaging alongside patient records, processing handwritten notes, understanding complex charts and diagrams, building visual coding assistants that can see a UI mockup and generate the code, and creating accessibility tools that describe images for visually impaired users.