Signals

A signal is a label on a span saying something happened in a conversation: the agent hit a rate limit, retried the same tool five times, refused a request, or the user gave up.

Oodle detects most signals as your traces arrive. That costs nothing per span and runs on all of them, so a signal has a real rate you can chart, alert on and compare between releases, rather than a sampled estimate.

Open Agent Observability → Evaluators → Signals (ap1, us1) to see which signals your traffic carries.

What a signal carries

Every signal writes these span attributes:

Attribute	Meaning
oodle.signal.execution	The leaf type, such as environment.rate_limited
oodle.signal.category	The family: interaction, execution.failure, execution.loop, environment or agent
oodle.signal.polarity	positive, negative or neutral
oodle.signal.source	Where it came from: detected at ingest, evaluated, or reported by the agent
oodle.signal.reason	The pattern that matched

The same values are labels on the oodle_genai_signal_total metric, so a trace filter and a metric query give the same answer.

Polarity is set when the signal is written, not looked up later. That matters for a signal like user satisfaction, where the direction comes from the sign of the value and no table could say it.

Signals your agent reports about itself

Some things only the agent knows. A missing credential looks like an ordinary clarifying question from the outside, and no external detector can tell the difference.

Your agent reports one by setting a span attribute. There is no SDK to install: every framework already emits spans, so this is a convention rather than a dependency.

span.set_attribute("oodle.signal.agent", "missing_context")
span.set_attribute(
    "oodle.signal.agent.detail",
    "no billing account on file for this customer",
)

oodle.signal.agent takes one of four categories:

Category	Use it when
missing_context	The agent lacks information or a credential it needs
repeatedly_broken_tool	A tool keeps failing in a way the agent cannot work around
capability_gap	The agent was asked for something it cannot do
complete_task_failure	The agent finished without completing the task

oodle.signal.agent.detail is free text explaining the specific case, kept to 256 characters. It rides along as the signal's reason.

Your own category names work too. An unrecognised name is kept and reported under the general agent.reported type, with the name you sent as the reason, so you can start with the four above and add your own as you learn what your agent runs into.

Give the agent a tool that sets these attributes, and say in its prompt when to reach for it. The report then arrives on the same trace as the conversation it describes, next to the tool calls and the model responses that led to it.

Signals from thumbs up and down

Feedback you already send through POST scores appears alongside detected signals, marked as instrumented. It needs no extra work: it is the same score you were already writing, on the screen where people look for behaviour.

Writing your own

Describe a behaviour in plain English and Oodle drafts a classifier for it, shows you what it matched in your own traces, and turns it into a signal once you are happy. See Agent Observability → Evaluators → Signals.

A new signal counts from the moment you turn it on. To find out what it would have caught before that, run a backfill from the Backfills tab: it reads past traffic and reports what the evaluator would have found, keeping those results separate from your live scores.