OpenLLMetry
OpenTelemetry-based observability for LLMs
Verdict
Best choice if your org is OpenTelemetry-native. Routes LLM traces into your existing Grafana/Datadog stack. Vendor-neutral but still maturing on evaluation tooling.
Other Observability & Evals
- Arize AIStable
ML and LLM observability platform with drift detection and evaluation
- BraintrustExperimental
End-to-end LLM evaluation and experimentation platform
- HeliconeStable
LLM gateway with logging, caching, and cost analytics
- LangfuseProduction
Open-source LLM engineering platform — traces, evals, prompts
- Phoenix (Arize)Stable
Open-source AI observability with embedding and RAG tracing
- PromptLayerStable
Prompt version control, A/B testing, and analytics platform