LangSmith is LangChain's observability and evaluation platform for LLM applications. It automatically traces all LangChain/LangGraph runs, capturing inputs, outputs, latency, token usage, and intermediate steps. Playground allows live prompt experimentation. Datasets management stores golden test cases. Evaluation runners compare prompt variants. Works with any LLM — not just LangChain — via direct SDK instrumentation.
Tracing non-LangChain code with LangSmith SDK.
Without tracing, debugging an LLM pipeline means adding print statements and running locally. LangSmith captures every step in production automatically. When a user reports a bad answer, find the trace in seconds.
When you find a bad production trace, one click sends it to a dataset. The dataset grows with real failures. Use it for offline regression testing. The offline/online eval cycle improves quality continuously.
"Does prompt A or prompt B work better?" Without LangSmith: manually test a few examples, guess. With LangSmith: run both on 200 golden samples, compare faithfulness and relevance scores statistically. Data-driven decisions.
Sign in to share your feedback and join the discussion.