Evaluator Dependencies

Evaluator dependencies let you chain evaluation rules into a directed acyclic graph (DAG). A dependent rule only runs on spans where all of its dependencies scored > 0, saving LLM costs by skipping spans that fail earlier, cheaper checks.

How it works

1. The eval-worker builds a DAG from each rule's dependsOnRuleIds field.
2. Rules are sorted into topological levels — level 0 has no dependencies, level 1 depends only on level 0, and so on.
3. After each rule runs, the eval-worker records which span IDs had any score greater than zero.
4. Before running a dependent rule, the worker filters candidates to only spans that passed all of that rule's dependencies (AND logic).
5. Each rule still applies its own dedup, sampling, and rate limits on the filtered set.

What counts as "passing"

A span passes a dependency if any numeric score returned by that rule is greater than zero. Categorical scores are ignored.

Example: code filter → LLM deep analysis

Code eval: has_output (level 0)
    ↓
LLM eval: Correctness (level 1)
    ↓
LLM eval: Deep analysis (level 2)

1. has_output is a code evaluator that returns 1 if the span has non-empty output, 0 otherwise. It runs on all matching spans.
2. Correctness depends on has_output. It only runs the LLM judge on spans where the code eval scored

   0 (i.e., spans that actually have output).

3. Deep analysis depends on Correctness. It only runs on spans that passed the correctness check.

This saves LLM costs because the cheap code evaluator filters out irrelevant spans before the expensive LLM calls happen.

Multi-dependency example

Code eval: has_output (level 0)
Code eval: is_english (level 0)
    ↓           ↓
LLM eval: Correctness (level 1)

Correctness depends on both has_output and is_english. A span must pass both code evaluators to be evaluated for correctness.

Setting up dependencies

1. Go to GenAI → Evaluators → New Evaluator (or edit an existing rule).
2. In the Depends on section, select one or more existing rules.
3. The API validates that the new dependency does not create a cycle. If it does, the save is rejected with an error.

Cycle prevention

The system blocks saving any rule that would create a dependency cycle. For example, if rule A depends on rule B, you cannot make rule B depend on rule A.

Cycle detection runs both server-side (at save time) and is enforced by the API, which returns a 400 error if a cycle is detected.

Deleting a dependency target

If you try to delete a rule that other rules depend on, the API returns a 409 Conflict error listing the dependent rule names. You must first remove the dependency from those rules before deleting the target.

Visualizing dependencies

Dependencies tab

The Dependencies tab on the Evaluators page shows the full DAG across all rules. Each node represents a rule with its name, type (Code or LLM), and status (Active or Paused). Edges show the dependency direction.

Click any node to open the evaluator details drawer.

Drawer subtree view

Inside the evaluator details drawer, the Dependencies tab shows the subtree for the focused rule — both its upstream dependencies and downstream dependents. The focused node is highlighted.

Best practices

• Same filters recommended: dependent rules should share the same span filters for best results. The eval-worker groups rules by filter set, so cross-group dependencies work but span overlap may be limited.
• Use code evals as first-pass filters: code evaluators are fast and cheap. Use them to filter out irrelevant spans before running LLM judges.
• Keep the DAG shallow: deeply nested dependencies add latency to each eval cycle. Two to three levels is usually sufficient.