Skip to main content

LogMetrics

LogMetrics convert log events into Prometheus metrics while Oodle ingests the logs. You can then chart the metrics in dashboards and use them in monitors. A monitor on a log metric is how you alert on logs. For the full alert flow, see Alerting on Logs and Traces.

LogMetrics work on structured logs and on plain-text logs. A regular expression can take a label value or a numeric value from any part of a text line. You do not need a log transform first.

Use LogMetrics to:

  • Count log lines that match a filter, for example failed logins or ERROR lines.
  • Count log lines per value of a field or a regex capture, for example per user, per tenant or per tool.
  • Record a number from a log line, for example a duration, a byte count or a queue size, as a counter, a gauge or a histogram.

How It Works​

  1. Logs enter the ingestion pipeline.
  2. The rule filter selects the matching log events.
  3. Each metric definition in the rule emits one sample for each matching event.
  4. Oodle attaches the rule labels to each sample.
  5. You query the metrics with PromQL in dashboards, in the metrics explorer and in monitors.
note

A rule applies only to logs that Oodle ingests after you save the rule. Oodle does not backfill metrics from logs that it ingested before. To answer a question about past logs, use a log search aggregation.

Example Walkthrough​

In the following video, we walk through how Log Metrics can be used to convert logs into alert-ready metrics.

Create a Rule​

You can create a rule in four ways:

  • UI: In the Oodle sidebar, open Logs, then LogMetrics. Click New LogMetrics.
  • API: Send the rule as JSON to the LogMetrics API. See API.
  • CLI: Run oodle log-metrics create -f rule.yaml. See CLI.
  • Terraform: Use the oodle_logmetrics resource. See Terraform.

Rule Schema​

A rule has a name, a filter, labels and one or more metric definitions.

{
"name": "checkout-errors",
"filter": {
"all": [
{ "field": "service", "operator": "is", "value": "checkout" },
{ "field": "level", "operator": "is", "value": "error" }
]
},
"labels": [
{ "name": "team", "value": "payments" },
{ "name": "route", "valueExtractor": { "field": "message", "regex": "path=(/[a-z/]+)" } }
],
"metricDefinitions": [
{ "name": "checkout_error_lines", "type": "log_count" }
]
}
KeyRequiredDescription
nameYesName of the rule.
filterNoSelects the log events that the rule uses. When you do not set a filter, the rule uses all log events.
labelsNoLabels that Oodle adds to each metric of the rule.
metricDefinitionsYesOne or more metrics that the rule emits.

Filter​

A filter is one of four forms. You can nest the forms.

FormJSONMatches when
Match{ "field": "...", "operator": "...", "value": "..." }The field matches the value.
All{ "all": [ <filter>, ... ] }All of the filters match (AND).
Any{ "any": [ <filter>, ... ] }One or more of the filters match (OR).
Not{ "not": <filter> }The filter does not match.

You can write a match inline, as in the table, or in a match object: { "match": { "field": "...", "operator": "...", "value": "..." } }. Both forms work at each level of a filter. Use only one form in a filter object.

A match has these keys:

KeyDescription
fieldTop-level log field name. Use only letters, digits and underscores, and do not start with a digit.
jsonPathOptional. A path such as user.id or items[0].sku into a field that holds an object or a JSON string.
operatorOne of the operators in the table below.
valueThe value to compare. Not used with exists.
OperatorMatches when
isThe field value, as a string, is equal to value.
containsThe field value contains value. This operator ignores case.
matches regexThe regular expression in value matches part of the field value. Add ^ and $ to match the full value.
existsThe field is present and is not null.

Labels​

Each label has a name and one of these value sources:

  • value: a static string. For example, "team": "payments".
  • valueExtractor: a value that Oodle reads from each log event.

A valueExtractor has these keys:

KeyDescription
fieldTop-level log field to read. Required.
jsonPathOptional. A path to a nested value in the field.
regexOptional. A regular expression. The label value is the text that capture group 1 matches.
lookupOptional. Maps the extracted value through a lookup table. It has tableId, matchColumn, valueColumn and an optional fallback ({ "field": "..." }) that Oodle uses when no row matches.

Oodle applies the keys in this order: field, then jsonPath, then regex, then lookup. When the regex does not match, the sample has no value for that label.

Metric Definitions​

KeyDescription
nameMetric name. Use the Prometheus name rules: letters, digits and underscores.
typelog_count, counter, gauge or histogram.
fieldThe log field that holds the number. Required for counter, gauge and histogram. Do not set it for log_count.
jsonPathOptional. A path to a nested number in the field.
regexOptional. A regular expression. The number is the text that capture group 1 matches.

The metric types are:

TypeEach matching log eventQuery with
log_countAdds 1 to a counter.increase() or rate()
counterAdds the extracted number to a counter. Use it for totals, for example bytes sent.increase() or rate()
gaugeSets the gauge to the extracted number. Use it for a level, for example queue size.the metric value, max_over_time(), avg_over_time()
histogramAdds the extracted number to a histogram. Use it for distributions, for example latency.histogram_quantile() on the _bucket series

When the extracted text is not a number, Oodle records 0 for that event. Make sure that the filter selects only the lines that contain the number. A good method is to add a matches regex filter that uses the same pattern as the metric definition.

Regular Expressions​

Regular expressions use the Rust regex syntax. You can test a pattern at rustexp.lpil.uk. Some useful forms:

PatternCaptures
user=(\S+)The text after user= up to the next space.
in (\d+) msThe digits in in 1250 ms.
took (\d+(?:\.\d+)?)sA decimal number such as 2.75 in took 2.75s.
(?i)status=(\w+)A word after status=, and the match ignores case.
"user_id":"([^"]+)"A JSON string value inside a text line.

Use (?:...) for groups that you do not want to capture, because Oodle uses only capture group 1.

In JSON, escape each backslash: the pattern in (\d+) ms is "in (\\d+) ms". In YAML, put the pattern in single quotes: 'in (\d+) ms'.

Metric Names​

Oodle adds the prefix oodle_logs_ to each metric name. If the name already starts with oodle_logs_, Oodle does not add the prefix again.

TypeDefinition nameSeries that you query
log_countlogin_failuresoodle_logs_login_failures
counterbytes_sentoodle_logs_bytes_sent
gaugequeue_depthoodle_logs_queue_depth
histogramtool_latency_msoodle_logs_tool_latency_ms_bucket (with le), oodle_logs_tool_latency_ms_sum, oodle_logs_tool_latency_ms_count

Oodle does not add a _total suffix to counters.

Each sample also has an oodle_host label. More than one ingestion host can emit the same series, so aggregate with sum by (...) to get one value for each label set. For example:

sum by (user) (increase(oodle_logs_login_failures[10m]))

The histogram buckets are the same for all histogram rules in an instance: 0.1, 0.3, 0.5, 1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096, 8192, 16384, 32768, 65536, 131072 and +Inf. These buckets give good resolution for durations in milliseconds. To use different buckets, contact Oodle support.

Each sample has the timestamp of its log event. When the event has no timestamp, Oodle uses the ingestion time.

Cardinality​

Each different set of label values makes a new time series. Keep the number of values for each label bounded:

  • Good labels: service, level, route template, status code, tool name, tenant, user of an internal tool.
  • Bad labels: request ID, trace ID, session ID, timestamp, full URL with IDs, free-form error text.
  • Write the regex to capture only the stable part. For example, capture /orders from /orders/12345 with path=(/[a-z_/]+).
  • Use a lookup table to map a high-cardinality ID to a small group, for example a customer ID to a plan tier.
  • To find the top values of a high-cardinality field, use a log search terms aggregation. Do not make a metric label for it.

Example: Count Failed Logins for Each User​

Plain-text log lines:

2026-10-07T10:15:02Z WARN auth Login failed for user [email protected]: bad password
2026-10-07T10:15:09Z WARN auth Login failed for user [email protected]: account locked

The log text is in the message field. This rule counts the failed logins and puts the user in the user label:

{
"name": "login-failures-by-user",
"filter": {
"all": [
{ "field": "message", "operator": "contains", "value": "Login failed for user" }
]
},
"labels": [
{
"name": "user",
"valueExtractor": { "field": "message", "regex": "Login failed for user (\\S+?):" }
}
],
"metricDefinitions": [
{ "name": "login_failures", "type": "log_count" }
]
}

Failed logins for each user in the last 10 minutes:

sum by (user) (increase(oodle_logs_login_failures[10m]))

A monitor on this query with the condition Above 5 sends one alert for each user. The alert has the user label, so the message can name the user with {{ $labels.user }}. See the full monitor example.

Example: Latency Histogram From Plain-Text Lines​

Plain-text log lines:

INFO agent tool=search_docs status=ok in 840 ms
INFO agent tool=run_query status=ok in 12450 ms
INFO agent tool=run_query status=error in 30012 ms

This rule records the duration as a histogram, with labels for the tool and the status:

name: agent-tool-latency
filter:
all:
- field: message
operator: matches regex
value: 'tool=\S+ .*in \d+ ms'
labels:
- name: tool
valueExtractor:
field: message
regex: 'tool=(\S+)'
- name: status
valueExtractor:
field: message
regex: 'status=(\w+)'
metricDefinitions:
- name: tool_latency_ms
type: histogram
field: message
regex: 'in (\d+) ms'

p95 latency for each tool, in milliseconds:

histogram_quantile(
0.95,
sum by (tool, le) (rate(oodle_logs_tool_latency_ms_bucket[5m]))
)

Number of calls for each tool in the last hour:

sum by (tool) (increase(oodle_logs_tool_latency_ms_count[1h]))

Number of calls that took more than 8192 ms (a bucket edge) in the last hour:

sum by (tool) (increase(oodle_logs_tool_latency_ms_bucket{le="+Inf"}[1h]))
-
sum by (tool) (increase(oodle_logs_tool_latency_ms_bucket{le="8192"}[1h]))

Count Lines Above an Exact Threshold​

A histogram counts only at its bucket edges. To count lines above an exact value, put the threshold in the filter regex and use log_count. This rule counts tool calls that took 10 seconds (10000 ms) or more:

name: slow-tool-calls
filter:
all:
- field: message
operator: matches regex
value: 'in [1-9]\d{4,} ms'
labels:
- name: tool
valueExtractor:
field: message
regex: 'tool=(\S+)'
metricDefinitions:
- name: slow_tool_calls
type: log_count
sum by (tool) (increase(oodle_logs_slow_tool_calls[15m]))

API​

Base path: https://<OODLE_INSTANCE>.api.oodle.ai/v1/api/instance/<OODLE_INSTANCE>/logmetrics. Send the headers X-OODLE-INSTANCE: <OODLE_INSTANCE> and X-API-KEY: <OODLE_API_KEY>. Get both values from Settings > API Keys.

MethodPathAction
GET/logmetricsList the rules.
POST/logmetricsCreate a rule. The body is the rule JSON.
GET/logmetrics/{id}Get a rule.
PUT/logmetrics/{id}Replace a rule.
DELETE/logmetrics/{id}Delete a rule.
curl -X POST \
"https://<OODLE_INSTANCE>.api.oodle.ai/v1/api/instance/<OODLE_INSTANCE>/logmetrics" \
-H "X-OODLE-INSTANCE: <OODLE_INSTANCE>" \
-H "X-API-KEY: <OODLE_API_KEY>" \
-H "Content-Type: application/json" \
-d @rule.json

The API checks each regular expression and each field name. It returns HTTP 400 with a message when a value is not valid.

CLI​

The Oodle CLI reads a rule from a JSON or YAML file:

oodle log-metrics list
oodle log-metrics create -f rule.yaml
oodle log-metrics get <id>
oodle log-metrics update <id> -f rule.yaml
oodle log-metrics delete <id>

Terraform​

Define rules with the oodle_logmetrics resource. The resource uses snake_case keys (value_extractor, json_path, metric_definitions). In HCL, each condition is in a match block. The provider sends the condition to the API in the inline form.

resource "oodle_logmetrics" "login_failures" {
name = "login-failures-by-user"

filter = {
all = [
{
match = {
field = "message"
operator = "contains"
value = "Login failed for user"
}
}
]
}

labels = [
{
name = "user"
value_extractor = {
field = "message"
regex = "Login failed for user (\\S+?):"
}
}
]

metric_definitions = [
{
name = "login_failures"
type = "log_count"
}
]
}

See the full schema in the Terraform registry.


Support

If you need assistance or have any questions, please reach out to us through: