GenAI semantic conventions, span attributes, metrics, propagation.

OpenTelemetry for AI — Semantic Conventions and LLM Spans

OpenTelemetry (OTel) is the standard observability framework, now extended with GenAI semantic conventions for LLM tracing. Key span attributes: gen_ai.system (openai/anthropic), gen_ai.request.model, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, gen_ai.response.finish_reason. OTel enables vendor-neutral LLM observability — same instrumentation, any backend (Jaeger, Tempo, Azure Monitor, LangSmith). The OpenInference specification extends OTel for RAG and agent traces.

Interactive Pipeline Flow

UnderstandImplementOptimise

🔧 Instrument — tracer setup

Setup: TracerProvider with OTLPSpanExporter. For LangChain: register LangChainInstrumentor(). For custom: instrument manually with span creation.

Architecture Diagrams

Manually instrumenting an LLM call with OTel.

Loading diagram...

Key Design Decisions

🔧OTel GenAI conventions enable vendor-neutral LLM observability

LangSmith and Phoenix use proprietary trace formats. OTel GenAI semantic conventions are a standard — the same span attributes work with Azure Monitor, Grafana Tempo, Jaeger, and any OTel-compatible backend. No vendor lock-in on your observability data.

🔗Trace propagation connects HTTP, DB, and LLM spans into one trace

Inject the W3C traceparent header when calling LLM APIs. Now your APM shows one trace: user HTTP request → DB query → LLM call → response. See exactly how much of total latency comes from LLM vs retrieval.

📊OTel metrics give you standardised LLM dashboards across all your services

gen_ai.client.token.usage and gen_ai.client.operation.duration are standard histogram metrics. Use the same Grafana dashboards and alerts for all LLM services. Export to any metrics backend without changes.

Discussion

OpenTelemetry for AI — Semantic Conventions and LLM Spans

Key Design Decisions

🔧OTel GenAI conventions enable vendor-neutral LLM observability

🔗Trace propagation connects HTTP, DB, and LLM spans into one trace

📊OTel metrics give you standardised LLM dashboards across all your services