How to trace and debug LLM calls with LangSmith

LangSmith is LangChain's observability platform for logging, tracing, and evaluating LLM applications. This guide covers setup, automatic tracing, custom traces, and using the dashboard to debug production issues.

Why LLM observability matters

LLMLLMLarge Language Model — a neural network trained on vast amounts of text data that can understand and generate human-quality language across a wide range of tasks.Learn more → applications are opaque by default. When a user reports a bad answer, you need to see the exact promptPromptThe input text sent to a language model — the question, instruction, or context that triggers a response.Learn more → that was sent, the model's response, which tools were called, what they returned, and how long each step took. Without tracing, debugging requires reproducing the issue from scratch — often impossible with production data.

LangSmith captures every LLM call and tool execution automatically when you use LangChain or LangGraph. For non-LangChain applications, the SDK provides a simple decorator and context manager API for manual instrumentation.

Set up LangSmith

Create a free account at smith.langchain.com. Create a project and copy your API key. Set environment variables: `export LANGCHAIN_TRACING_V2=true; export LANGCHAIN_API_KEY=ls__...; export LANGCHAIN_PROJECT=my-project`. That's it — any LangChain code now automatically logs to LangSmith.

Free tier limits: 5000 traces per month. This is generous for development but will be exceeded in production. The Plus plan ($39/month) includes 10M traces. For self-hosted deployments, LangSmith is open-source and can be run on your own infrastructure.

Read and navigate traces

Each trace shows the full execution tree: the top-level chain, sub-chains, individual LLM calls with the exact prompt and response, tool calls with inputs and outputs, and latencyLatencyThe delay between sending a request to an LLM and receiving the first token of the response, often measured as Time to First Token (TTFT).Learn more → at each step. Click any node to expand it and see the full content.

Use the filter UI to find specific traces: filter by date range, model name, latency, tokenTokenThe basic unit of text that an LLM processes — roughly corresponding to a word or word-piece. Models read input and produce output in tokens, which is also how API usage is measured and billed.Learn more → count, or custom tags. For multi-user applications, tag traces with the user ID: `from langsmith import traceable; @traceable(metadata={'user_id': user.id}) def my_function(): ...`. Then filter by user in the dashboard.

Instrument non-LangChain code

Use the `@traceable` decorator for any function you want to trace: `from langsmith import traceable; @traceable(name='classify_document', tags=['production']) def classify(text: str) -> str: response = openai.chat.completions.create(...); return response.choices[0].message.content`. The decorator automatically captures inputs, outputs, and timing.

For manual tracing with more control: `from langsmith import Client; client = Client(); run = client.create_run(name='my_run', inputs={'prompt': prompt}); # ... do work ...; client.update_run(run.id, outputs={'result': result}, end_time=datetime.now())`.

Evaluate with LangSmith datasets

Create a dataset of test prompts and expected outputs in the LangSmith UI. Run your application against the dataset with an evaluator: `from langsmith.evaluation import evaluate; results = evaluate(my_app, data='my-dataset', evaluators=[exact_match, llm_judge])`. The dashboard shows pass rates and diffs between runs.

Use datasets to catch regressions when you update prompts or switch models. Run the evaluation before and after the change and compare the scores. This turns LLM evaluation from an ad-hoc process into a repeatable CI check.