Why you need a prompt library
As you build LLMLLMLarge Language Model — a neural network trained on vast amounts of text data that can understand and generate human-quality language across a wide range of tasks.Learn more → applications, prompts proliferate across scripts, notebooks, and application code. Without organisation, you end up with duplicate prompts, no way to track what changed when something breaks, and no mechanism to share good prompts across the team.
A promptPromptThe input text sent to a language model — the question, instruction, or context that triggers a response.Learn more → library treats prompts like code: each prompt has a name, a version, tests, and a changelog. When a model update changes prompt behaviour, you catch it in your test suite rather than in production.
Choose a storage format
The simplest approach is a YAML file per prompt: slug, version, content (the prompt text), model, parametersParametersThe numerical weights inside a neural network that are learned during training — the 'knowledge' of the model, measured in billions for modern LLMs.Learn more → (temperatureTemperatureA parameter controlling the randomness of model outputs — lower values produce more focused, deterministic responses; higher values produce more creative, varied text.Learn more →, max tokensMax TokensThe maximum number of tokens an LLM can generate in a single response, separate from the context window that determines input capacity.Learn more →), and test_cases (input/expected output pairs). Store them in a `prompts/` directory in your repository.
For teams, a dedicated prompt management platform (LangSmith, Promptfoo, PromptLayer) adds a UI for editing, A/B testing, and tracking performance over time. These are worth the setup cost once you have more than 20 prompts in production.
Version and test your prompts
Use semantic versioning for prompts: 1.0.0 for the initial version, 1.0.1 for minor fixes, 1.1.0 for changes that alter the output format, 2.0.0 for complete rewrites. Reference prompts by version in your application code: `get_prompt('summarise-article', version='1.2.0')`.
Write test cases for each prompt: a set of input-output pairs that define correct behaviour. Run them automatically in CI using an LLM-as-judge pattern — have a second model evaluate whether the output matches the expected criteria. Set a threshold (e.g. 90% pass rate) as a quality gate.
Practical structure for a prompt file
A good prompt YAML includes: name, description (what it does and when to use it), model (which model it was tested with), parameters, system_prompt, user_prompt_template (with placeholders like `{{document}}`), and examples with expected outputs.
Keep prompt logic in templates, not in Python string interpolation. Templates are readable, diffable, and editable by non-engineers. Use Jinja2 or Mustache syntax — both are well-supported and produce clean diffs in version control.