Metrics reference
You can use the Imply Polaris Metrics API to emit ingestion and query performance metrics into third-party monitoring tools.
Polaris exposes the following types of metrics. For more information on metric types, refer to the official OpenMetrics documentation.
- Gauge: A measurement that can increase, decrease, or stay constant over time.
- Summary: A measurement used to sample observations, such as request duration and response sizes.
This topic contains a list of performance metrics and their labels available in the Metrics API.
Labels
A label is a key-value attribute of a metric. Labels differentiate the characteristics of the resource that is being measured.
All Polaris metrics share the region
label. It represents the deployment region of the project for which metrics are being reported.
Some metrics may include the following additional labels:
job_id
: The ID of the ingestion job.quantile
: The quantile measurement, where0.5
is the median,0.98
corresponds to the 98th percentile, and+Inf
correspond to the maximum value of the metric.table
: The table that the metric applies to.tables
: The list of queried tables that the metric applies to.type
: The type of ingestion.
Query metrics
Polaris exposes the following query metrics.
query_count
The total number of queries issued over one minute.
- Type: gauge
- Labels:
tables
,region
query_time_ms
The time, in milliseconds, that it took to complete a query.
- Type: summary
- Labels:
tables
,region
,quantile
Polaris provides query_count
and query_time_ms
metrics for tables that have been queried. If you haven't queried any tables, these metrics are not available.
Configure your monitoring tools to zero fill missing data in the UI.
A query that omits the table name—for example, SELECT 1234
—or fails SQL planning—for example, SELECT * FROM table WHERE nonExistentColumn='abc'
—has its tables
label set to tables=[]
.
The order of elements in the tables
list is not guaranteed. For example, if you have two queries SELECT * from A, B
and SELECT * from B, A
, the metrics for these queries can be split across two lists tables=[A, B]
and tables=[B, A]
.
Kafka ingestion metrics
Polaris exposes the following metrics for Apache Kafka and push streaming ingestion jobs.
ingest_job_count
The total number of ingestion jobs.
- Type: gauge
- Labels:
table
,region
,type
ingest_kafka_lag
The total lag between the offsets consumed by the Kafka streaming job and offsets in the Kafka topic across all partitions.
The lag reflects the time from when Polaris returns an HTTP 200
status code and the time the data is ingested into Polaris. The lag doesn’t account for the time between an application sending a request and Polaris responding with a 200
.
- Type: summary
- Labels:
table
,region
,job_id
,quantile
ingest_events_processed
The number of events successfully processed during ingestion.
- Type: gauge
- Labels:
table
,region
,job_id
ingest_events_thrownAway
The number of events rejected during ingestion because they are either null, filtered out, or outside the retention period.
- Type: gauge
- Labels:
table
,region
,job_id
ingest_events_unparseable
The number of events rejected because the events are unparseable.
- Type: gauge
- Labels:
table
,region
,job_id
Table metrics
Polaris does not compute table metrics every minute. These metrics are only emitted when calculated.
Configure your monitoring tools to use the previous value instead of 0
.
table_bytes_compacted
The total size, in bytes, of the compacted data in the table.
- Type: gauge
- Labels:
table
,region
table_intervals
The total number of intervals in the table.
- Type: gauge
- Labels:
table
,region
table_intervals_compacted
The number of intervals in the table that have been compacted.
- Type: gauge
- Labels:
table
,region
table_size
The total size of the table in bytes.
- Type: gauge
- Labels:
table
,region