What GPT-4o vision can do
GPTGPTGenerative Pre-trained Transformer — the model architecture and family name behind OpenAI's most famous models, from GPT-2 to GPT-5.Learn more →-4o's vision capabilities go far beyond simple image captioning. It can read text from images (OCR), interpret charts and graphs, analyse screenshots, understand diagrams and schematics, compare multiple images, and reason about spatial relationships — all in a single API call.
Practical applications: automated document processing, screenshot-to-code conversion, chart data extraction, accessibility descriptions for UI elements, quality control in manufacturing, and medical image analysis for supported use cases.
Send images via URL
The simplest approach is to reference a publicly accessible image URL in the message content: `messages=[{"role": "user", "content": [{"type": "image_url", "image_url": {"url": "https://example.com/chart.png"}}, {"type": "text", "text": "What are the values in this chart?"}]}]`.
Use `detail: 'high'` for images that require reading small text or fine details: `{"image_url": {"url": "...", "detail": "high"}}`. High detail mode tiles the image into 512×512 chunks and processes each — it costs more tokens but produces significantly better results for documents and dense charts.
Send images via base64
For local files or private images, encode to base64: `import base64; with open('image.jpg', 'rb') as f: data = base64.b64encode(f.read()).decode()`. Then: `{"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{data}"}}`.
Supported formats: JPEG, PNG, GIF, WebP. Maximum image size is 20MB. For screenshots and UI analysis, use PNG to preserve crisp text rendering — JPEG compression artifacts reduce OCR accuracy.
Practical: extract data from charts
Prompt engineeringPrompt EngineeringThe practice of crafting inputs, instructions, examples, and formatting to guide language models toward producing better, more accurate outputs.Learn more → matters for data extraction. Instead of 'What does this chart show?', use: 'This is a bar chart. List every bar with its exact label and approximate value. Format your response as a JSON array of {label, value} objects.'
For financial charts, add: 'Values are in millions of dollars. Include units in the value field. If a value is not clearly readable, indicate uncertainty with a range (e.g. 45-50M).' Specific, structured prompts produce machine-parseable output that generic prompts do not.
Practical: screenshot to code
Send a UI screenshot and ask the model to generate the implementation: 'Here is a screenshot of a web form. Write the HTML and CSS to replicate this UI as closely as possible. Use Tailwind CSS classes. Include all visible form fields and the submit button.'
For better results, use high detail mode and include the tech stack context: 'We are using ReactReActA prompting pattern that alternates between reasoning steps and tool-use actions, enabling models to think through problems, take actions, observe results, and iterate.Learn more → with TypeScript and Tailwind CSS. Generate a React component.' GPT-4o produces surprisingly accurate UI code from screenshots, especially for clean, standard UI patterns.