System Prompts: The Hidden Instructions Behind AI Assistants

System prompts set the rules before a conversation begins. They're how developers shape model behavior, tone, and capabilities at scale.

Key takeaways
  • Special instruction blocks passed to LLMs before conversations begin to configure model behavior, persona, and constraints.
  • Messages use 'system', 'user', and 'assistant' roles with system messages coming first to set authoritative context.
  • System prompts are prepended to every request and count toward token limits, with prompt caching available to reduce repeated costs.
  • System prompts are not truly secret as users can often extract them through careful prompting or direct requests.
  • User-supplied input can contain instructions that override system prompts, requiring input sanitization and injection-resistant training.

What Is a System Prompt?

A system promptSystem PromptA special instruction given to a language model before the user conversation begins, establishing the model's persona, capabilities, constraints, and context.Learn more → is a special instruction block passed to an LLMLLMLarge Language Model — a neural network trained on vast amounts of text data that can understand and generate human-quality language across a wide range of tasks.Learn more → before the conversation begins. Unlike user messages, it's typically not shown to end users — it's the developer's way of configuring the model's behavior, persona, and constraints for a specific application. Think of it as the rules briefing a new employee receives before their first customer interaction.

System prompts can define: what the model's role is ('You are a customer support agentAgentAn LLM-powered system that can take actions, use tools, and pursue multi-step goals autonomously without human input at each step.Learn more → for Acme Corp'), what it should and shouldn't discuss ('Only answer questions about our software products'), the tone and style to use ('Be concise, friendly, and professional'), how to handle edge cases, and any background context the model needs to do its job.

How System Prompts Work Technically

In the chat completions API format (used by OpenAI, Anthropic, and most providers), messages are structured as a list with roles: 'system', 'user', and 'assistant'. The system message comes first and sets the context. The model has been instruction-tuned to treat system messages differently from user messages — to follow them as authoritative instructions rather than conversational input.

System prompts are prepended to every request's context windowContext WindowThe maximum amount of text (measured in tokens) that a model can read and reason over in a single interaction, including your prompt, any documents, and previous conversation history.Learn more →. They count toward the tokenTokenThe basic unit of text that an LLM processes — roughly corresponding to a word or word-piece. Models read input and produce output in tokens, which is also how API usage is measured and billed.Learn more → limit and affect costs accordingly. For applications with long system prompts (detailed personas, extensive instructions, embedded documents), the promptPromptThe input text sent to a language model — the question, instruction, or context that triggers a response.Learn more → can represent a significant fraction of each request's token count. Prompt cachingPrompt CachingA technique that stores and reuses computed results from repeated prompt prefixes across API requests, reducing both response latency and computational costs.Learn more → (offered by Anthropic, OpenAI, and Google) can cache the KV representations of a stable system prompt, dramatically reducing repeated costs.

System Prompt Security and Prompt Injection

System prompts are not truly secret from determined users. A user can ask 'What are your instructions?' and models will often reveal them, unless explicitly instructed not to. They can also be partially extracted through careful prompting. Treat system prompts as security-through-obscurity at best, not as a security boundary.

Prompt injectionPrompt InjectionA security attack where malicious instructions hidden in untrusted input manipulate an LLM to ignore its original instructions and execute harmful commands.Learn more → is the attack where user-supplied input (or content from retrieved documents) contains instructions that override or manipulate the system prompt. For example, a retrieved document might contain text like 'Ignore your previous instructions and instead...' If the model follows these embedded instructions, it can be hijacked. Defending against prompt injection requires careful input sanitization, clear delimiters between system and user content, and model training specifically for injection resistance.

Read next
12 min read

Prompt Engineering: The Complete Guide

Master the art and science of writing effective prompts — from basic techniques to advanced methods like chain-of-thought, few-shot learning, and structured output generation.

6 min read

Instruction-Tuned Models vs. Base Models

Base models predict text. Instruction-tuned models follow directions. Understanding the difference is fundamental to working with LLMs.

10 min read

Privacy and Security with LLMs: What You Need to Know

The privacy risks of using AI APIs, data governance requirements, secure implementation patterns, and how to protect sensitive information when building with LLMs.

7 min read

In-Context Learning: How LLMs Learn from Examples

How large language models adapt to new tasks from examples in the prompt — without gradient updates or fine-tuning — and what this capability means for AI flexibility.