Haystack

Haystack's OpenTelemetry connector traces the pipeline and each component; the OpenAI instrumentation traces the model calls.

New to Oodle?

Oodle is a managed observability platform for metrics, logs, traces and agent traces. It ingests OpenTelemetry natively, so the snippet on this page is the complete setup.

Each trace shows the full transcript, the token counts and cost of every call, the agent structure (runs, steps and tool calls) and Signals: labels for loops, rate limits, refusals and tool failures, detected as the trace arrives.

Sign up for free to get an instance ID and an API key, or see the Agent Observability overview first.

You will need:

• OODLE_INSTANCE: your Oodle instance ID (ap1, us1)
• OODLE_API_KEY: an Oodle API key (ap1, us1)
• OTLP_ENDPOINT: your OTLP collector domain, shown on the tile

Open the Haystack tile on the ap1, us1 page to get these filled in for you, or let an agent do the setup with /oodle-onboarding set up the llm_observability_haystack integration.

Install

Install Haystack with its OpenTelemetry integration and the OpenAI OTel instrumentation:

pip install haystack-ai opentelemetry-haystack \
  opentelemetry-instrumentation-openai \
  opentelemetry-exporter-otlp-proto-http \
  opentelemetry-sdk

Instrument

Initialize OpenTelemetry, activate the OpenAI instrumentor, and add the OpenTelemetry connector to the pipeline. Haystack then traces the pipeline and each component; the OpenAI instrumentor traces the model calls:

from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.connectors.opentelemetry import OpenTelemetryConnector
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.instrumentation.openai import OpenAIInstrumentor
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.trace import set_tracer_provider


def setup_opentelemetry():
    resource = Resource.create({"service.name": "my-llm-app"})

    tracer_provider = TracerProvider(resource=resource)
    tracer_provider.add_span_processor(
        BatchSpanProcessor(OTLPSpanExporter())
    )
    set_tracer_provider(tracer_provider)


setup_opentelemetry()

# Puts the prompt, completion and token counts on the model calls.
OpenAIInstrumentor().instrument()

pipeline = Pipeline()
# Traces the pipeline run and every component in it. It connects
# to nothing; adding it is enough.
pipeline.add_component("tracer", OpenTelemetryConnector("my-pipeline"))
pipeline.add_component(
    "prompt",
    ChatPromptBuilder(
        template=[ChatMessage.from_user("Answer in one sentence: {{question}}")],
        required_variables=["question"],
    ),
)
pipeline.add_component("llm", OpenAIChatGenerator(model="gpt-4o-mini"))
pipeline.connect("prompt.prompt", "llm.messages")

result = pipeline.run({"prompt": {"question": "What is the capital of France?"}})
print(result["llm"]["replies"][0].text)

Environment

Point the application at Oodle:

# OTLP endpoint (points straight at Oodle)
export OTEL_EXPORTER_OTLP_ENDPOINT=https://<OTLP_ENDPOINT>

export OTEL_EXPORTER_OTLP_HEADERS="X-API-KEY=<OODLE_API_KEY>,X-OODLE-INSTANCE=<OODLE_INSTANCE>"

# Compress the export: prompt payloads are large
export OTEL_EXPORTER_OTLP_COMPRESSION=gzip

# Put component inputs and outputs on the Haystack spans
export HAYSTACK_CONTENT_TRACING_ENABLED=true

Verify

Run your application, then open ap1, us1. Spans carry gen_ai.* attributes: the model, token counts, and the prompt and response content. Click a trace for the Transcript, the waterfall, and the cost breakdown.

If nothing arrives, check that the exporter can reach https://<OTLP_ENDPOINT> and that the instance and key are set: the OTLP gateway answers 401 without them.