Imply Polaris supports
LATEST_BY in input expressions to ingest
the earliest or latest value from a string input field.
Consider a use case in which you want to combine rows with the same timestamp
and dimension values, but for each bucket, you want to store the earliest value of a certain input field.
To do this, you create a measure in an aggregate table
and use the
EARLIEST_BY aggregation function in the
input expression for the measure.
This topic provides a description of
LATEST_BY and walks you through an example of
using these functions during ingestion.
You can use the following functions in input expressions.
|For the earliest time value in |
|For the latest time value in |
Keep the following details in mind when using
LATEST_BY in an input expression:
LATEST_BYonly work on string input fields. Applying these functions on a non-string input results in an invalid expression.
timestampExprshould correspond to the same timestamp value of the primary timestamp,
maxBytesPerStringto the number of bytes to allocate for aggregating strings. Polaris truncates any string beyond this limit; however, too high a value may lead to wasted memory. A reasonable value for
maxBytesPerStringis 1000 or 1024.
The following example shows how to use input expressions during ingestion to accomplish the following:
- Aggregate data by
- Populate the
first_IPcolumn based on the earliest occurring
srcIPvalue of the aggregated values.
- Populate the
last_IPcolumn based on the latest occurring
dstIPvalue of the aggregated values.
This example is based on the network flow event data used in the rollup example.
Follow the steps from the rollup example, but do not start the ingestion.
On the Map source to table page, add a string measure named
first_IPwith the following input expression:
EARLIEST_BY("srcIP", TIME_PARSE("timestamp"), 1000)
Add a second string measure named
last_IPwith the following input expression:
LATEST_BY("dstIP", TIME_PARSE("timestamp"), 1000)
Your table should look similar to the following. The
lhsfield lists the timestamp value in milliseconds, and the
rhsfield lists the first or last value.
Click Start ingestion.
For more information, see the following topics: