Datasets & Experiments

Datasets let you organize test cases for systematic evaluation of your LLM application. Each dataset contains input/expected-output pairs that you can run through an LLM with attached evaluators to measure quality at scale.

Navigate to Agent Observability → Datasets in the sidebar (ap1, us1).

Creating a Dataset

1. Click + New Dataset
2. Enter a name (e.g. sms-dataset-v1)
3. Optionally upload a CSV file to import items
4. Click Create

note

Creating and managing datasets requires the Editor or Admin role.

CSV Import

You can bulk-import items from a CSV file (up to 50 MB). The CSV must include an input column (exact case). Optional columns:

Column	Description
input	The user query or prompt (required)
expected_output	The ideal response
metadata	Additional context as JSON

A preview of the first few rows is shown before import. Partial imports are supported — if some rows fail, the successful rows are still added and a summary toast shows the error count.

Adding Items

From a dataset's detail page, click + New Item to add entries manually or via CSV.

Manual Entry

Add one or more rows with Input (required), Expected Output, and Metadata fields. Click + Add another row to add multiple items at once.

From a Trace

On any trace detail page, click Add to Dataset. Oodle splits the conversation into input and expected output and adds it to the dataset you select. This is useful for capturing real production examples as test cases.

Running Experiments

Experiments run your dataset items through an LLM and optionally score the outputs with evaluators.

1. Open a dataset and click Run Experiment
2. Configure the experiment:

Field	Description
LLM Connection	Which provider credentials to use (configure here)
Model	The model to run items against
Evaluators	One or more active evaluator rules to score outputs
Prompt	Optional prompt template to wrap inputs (by label or version)

3. Click Run Experiment

The experiment runs asynchronously. Results appear in the Runs tab.

tip

If you don't have an LLM connection configured yet, the form links to Settings → LLM Connections where you can add one.

Viewing Results

The Runs tab shows all experiment runs with their status, LLM connection, evaluator, prompt reference, and creation time.

Click any run to open a detail drawer showing:

• Run metadata — status, connection, model, evaluators, prompt, and any errors
• Per-item results — a table with Input, Metadata, Expected Output, Actual Output, and score badges from each evaluator

While an experiment is running, the drawer polls for new results every few seconds and shows a progress indicator.

Run Statuses

Status	Meaning
Pending	Queued, not yet started
Running	Processing dataset items
Completed	All items processed
Failed	An error stopped the run