LangSmith Gets Full OTel Support: What It Means

The Observability Problem Nobody Talks About Honestly

Building an LLM application is one thing. Knowing what it's actually doing at 2 AM when something goes sideways is another problem entirely.

Traditional observability tools were built for a world where 'something broke' meant an exception was thrown or a response time spiked past a threshold. They're good at that. But an LLM application can fail in ways that never trigger an alert — the model drifts toward a weird tone, a multi-step agent loop quietly skips a tool call, or a retrieval step returns documents that are technically relevant but practically useless. No exception. No timeout. Just a subtly wrong answer that your users notice before your monitoring does.

This is why the question of how you collect and route telemetry data from LLM applications matters so much more than it did for traditional services. And it's why LangSmith's move to ship full end-to-end OpenTelemetry support in its SDK is worth paying attention to — not because it's flashy, but because it quietly solves a real infrastructure headache for teams running complex, multi-service AI systems.

What Actually Changed

Here's the thing about this update that the announcement buries a little: LangSmith already accepted OpenTelemetry traces. It could ingest them on the backend. What it couldn't do was emit them natively from the SDK itself. That meant the OTel pipeline was half-built. You could receive OTel data, but generating it from your LangChain or LangGraph code required extra wiring that most teams didn't bother with.

Now the SDK handles the whole pipeline. Your LangChain application generates traces, the LangSmith SDK converts and ships them in the OTel standard format, and those traces land wherever you've pointed your collector — LangSmith's own dashboard, Datadog, Grafana, Jaeger, or all of the above simultaneously. That last part is the real unlock for teams that already have an observability stack and don't want to maintain a separate silo just for their AI layer.

Getting started is deliberately minimal. Install the package with OTel support:

pip install 'langsmith[otel]'
pip install langchain

Then set three environment variables:

LANGSMITH_OTEL_ENABLED=true
LANGSMITH_TRACING=true
LANGSMITH_API_KEY=your_key_here

That's it. Your existing LangChain code doesn't need to change. The instrumentation happens automatically. For teams that have been avoiding observability setup because it felt like a project unto itself, this is a meaningful reduction in friction.

Why OTel Is a Good Fit for LLM Tracing (and Where It Gets Complicated)

OpenTelemetry's core value proposition is vendor neutrality. It's an open standard, which means the data format isn't owned by any single observability vendor. You instrument once and you can route that telemetry anywhere. For conventional microservices, this has been a genuine quality-of-life improvement — teams stopped being locked into whichever APM tool they chose three years ago.

For LLM applications, the fit is mostly good but not perfect. OTel was designed around the concept of spans — discrete units of work with a start time, end time, and a set of attributes. That maps reasonably well onto a chain execution: each LLM call, each tool invocation, each retrieval step can be its own span. Distributed tracing across microservices works exactly as you'd expect, with context propagation linking spans from different services into a single coherent trace.

The complications show up at the edges. LLM outputs are stochastic — the same prompt doesn't always produce the same response, and 'correctness' isn't a binary you can encode in a span attribute. The emerging OpenTelemetry GenAI semantic conventions are trying to standardize how things like model name, token counts, and prompt/response content get attached to spans, but those conventions are still evolving. It's worth asking, before you commit to any OTel-based setup, whether the platform you're sending traces to actually understands those conventions or just treats your LLM spans as generic HTTP calls with extra attributes.

LangSmith's own dashboard is built specifically for LLM traces, so it handles the nuances — token usage, evaluation scores, intermediate reasoning steps — in ways that a general-purpose tool like Grafana won't do out of the box. That's not a knock on Grafana; it's just a different tool for a different job.

The Performance Trade-off Is Real, and You Should Care

LangSmith is upfront about this: the OTel format carries more overhead than their native tracing format. The native format was designed specifically for LLM data patterns — it's leaner, it supports real-time tracing with pending run states (so you can watch a long agent loop execute step by step), and it has a smaller memory footprint in the SDK.

The OTel format is more general-purpose. That generality costs something. How much it costs depends on your workload, but if you're running a high-throughput production system where every millisecond of added latency compounds, you'll want to benchmark both formats under realistic load before committing.

The practical guidance is straightforward: if LangSmith is your only observability destination and you have no plans to route traces anywhere else, stick with native tracing. You get better performance and features like pending run visibility that OTel doesn't support. If you need traces in Datadog for your SRE team and in LangSmith for your ML team, the OTel integration is the right call — just accept that you're paying a small performance tax for the flexibility.

The Questions This Update Doesn't Answer

A few things worth thinking through before you wire this into production.

First, trace volume. An agent that makes twenty tool calls per user request generates a lot of spans. At any meaningful scale, that's a storage and cost problem. OTel has sampling strategies — head-based, tail-based — that let you capture a representative subset of traces rather than every single one. The LangSmith integration doesn't spell out how sampling is configured or what the defaults are. For a small application this doesn't matter. For a system handling thousands of requests per hour, you'll want to understand this before your observability bill becomes uncomfortable.

Second, data privacy. Traces from LLM applications often contain the actual prompts and responses — which means they can contain sensitive user data. Shipping those traces through an OTel pipeline to multiple destinations means that data flows through more systems. Whether LangSmith offers any built-in masking or redaction for sensitive fields in the OTel path isn't clear from the current documentation. If your application handles anything regulated — healthcare, finance, legal — this is a conversation to have before enabling the integration.

Third, framework scope. The integration is explicitly for LangChain and LangGraph. Teams using LlamaIndex, or calling the OpenAI SDK directly, or building on other frameworks aren't covered here. OTel instrumentation libraries exist for many of these, but the seamless LangSmith integration described in this release is LangChain-specific. Worth knowing if your stack is mixed.

The Bigger Picture

There's a broader pattern worth naming. The AI tooling ecosystem is maturing, and part of that maturation is AI-specific tools starting to speak the same language as the broader infrastructure world. For a long time, LLM observability was a separate island — you had your model metrics over here and your infrastructure metrics over there, and getting a unified view required custom glue code or a lot of tab-switching.

OTel support in LangSmith is a step toward making the AI layer a first-class citizen in existing observability stacks. Your SRE team can use the tools they already know. Your AI team gets the LLM-specific visibility they need. The two don't have to be completely separate concerns anymore.

That's genuinely useful progress. Not because it's technically unprecedented — OTel instrumentation has existed for plenty of frameworks — but because it removes the excuse for AI teams to skip proper observability setup. 'It's too much work to integrate with our existing stack' gets harder to argue when the integration is three environment variables and a pip install.

The teams that will benefit most are the ones already running mature observability infrastructure who've been treating their LLM layer as a black box because wiring it up felt like a separate project. For them, this is the bridge they've been waiting for.