Skip to main content

Confluent Cloud

Oodle integrates with Confluent Cloud to pull metrics from the Confluent Cloud Metrics API. You provide a Cloud API key and the resources you want to watch, and Oodle collects Kafka cluster and connector metrics and makes them available in your instance for querying, dashboards, and alerting.

Prerequisites

  • A Confluent Cloud account
  • A Confluent Cloud Cloud API key with the MetricsViewer role, scoped to your organization or environment (not to a single Kafka cluster)
  • The IDs of the Kafka clusters (lkc-...) and, optionally, connectors (lcc-...) you want to monitor
  • An Oodle account (navigate to ap1, us1 to start setup)

Setup

1. Create a Confluent Cloud API key

In the Confluent Cloud console, open API keys and create a Cloud API key scoped to your organization or environment. Grant it the MetricsViewer role. A key scoped to a single Kafka cluster does not work with the Metrics API, so choose the organization or environment scope.

2. Find your resource IDs

Collect the IDs of the resources you want to monitor:

ResourceWhere to find itID format
Kafka clusterCluster Settingslkc-...
ConnectorData Integration > Connectorslcc-...

3. Add the account in Oodle

  1. Open the Confluent Cloud tile in Oodle (ap1, us1), or click Settings in the left sidebar and select the Confluent Cloud tile.
  2. Click Add account (the + button) and give the account a name (for example production). The name identifies this set of credentials in the list of accounts.
  3. Paste the API Key and API Secret you created.
  4. Enter the Cluster IDs (lkc-...) and, optionally, the Connector IDs (lcc-...) to monitor. Separate multiple IDs with commas.
  5. Click Save. Oodle validates the credentials against the Confluent Metrics API and begins collecting metrics within a few minutes.

Verification

Once setup is complete, verify the integration is working:

  1. In the Confluent Cloud tile, confirm the account status shows as Receiving.
  2. Go to ap1, us1 and search for metrics prefixed with confluent_ to confirm data is flowing.
  3. In the tile, open Visualize Data > View Dashboards and open the Confluent Cloud Overview dashboard.

Metrics collected

Metrics are collected per configured cluster and connector. Kafka metrics carry a kafka_id label, connector metrics carry a connector_id label, and per-topic metrics carry a topic label.

Kafka cluster metrics

MetricDescription
confluent_kafka_server_received_bytesBytes produced to the cluster
confluent_kafka_server_sent_bytesBytes consumed from the cluster
confluent_kafka_server_received_recordsRecords produced to the cluster
confluent_kafka_server_sent_recordsRecords consumed from the cluster
confluent_kafka_server_retained_bytesBytes retained by the cluster
confluent_kafka_server_active_connection_countActive client connections
confluent_kafka_server_partition_countPartitions on the cluster
confluent_kafka_server_request_countRequests by type
confluent_kafka_server_consumer_lag_offsetsConsumer group lag, by topic and group
confluent_kafka_server_cluster_load_percentEstimated cluster load

Connector metrics

MetricDescription
confluent_kafka_connect_sent_recordsRecords sent by the connector
confluent_kafka_connect_received_recordsRecords received by the connector
confluent_kafka_connect_sent_bytesBytes sent by the connector
confluent_kafka_connect_received_bytesBytes received by the connector
confluent_kafka_connect_dead_letter_queue_recordsRecords routed to the dead letter queue

Dashboards

Oodle provides an out-of-the-box Confluent Cloud Overview dashboard covering cluster throughput, connections, partitions, request mix, consumer lag, and connector activity. It is imported automatically when you save an account, and you can open it from Visualize Data > View Dashboards in the tile.

Data freshness

Confluent Cloud reports metrics at a one-minute resolution, and Oodle collects at the same one-minute granularity. New data points typically appear in Oodle within a few minutes of being recorded, in line with the Confluent Metrics API's own publication schedule.

Troubleshooting

IssueResolution
Save fails with an authentication errorConfirm the API key has the MetricsViewer role and is scoped to the organization or environment, not a single cluster
No metrics after a few minutesVerify the Cluster IDs (lkc-...) and Connector IDs (lcc-...) are correct and belong to the same organization as the API key
A metric is emptyThe cluster or connector may not currently be producing that metric (for example, consumer lag appears once a consumer group is active)

Support

If you need assistance or have any questions, please reach out to us through: