API docs by Redocly

Imply Polaris API (legacy)

Download OpenAPI specification:Download

You can use the Imply Polaris API to programmatically manage your tables, files, and ingestion tasks. See API overview for information and examples on how to get started.

The legacy APIs documented in this reference are provided for backwards compatibility and will be removed by the end of September 2024. We strongly recommend updating API requests to use the v1 version that includes the project ID in the base URL. For more information, see Migration guide for deprecated APIs.

For the latest version, see Polaris API reference.

Connections v2 (deprecated)

Manage connections to external sources of data.

List all connections

Returns all existing connections.

Authorizations:
apiKeyAuthbearerAuth

Responses

Response samples

Content type
application/json
{
  • "values": [
    ]
}

Create a connection

Creates a new connection.

For AWS-based connections, ensure you have the correct roles and permissions in AWS Identity and Access Management (IAM). See Access to AWS resources.

Authorizations:
apiKeyAuthbearerAuth
Request Body schema: application/json

A connection to be created.

bootstrapServers
required
string

List of host and port pairs used for establishing the initial connection to the Kafka cluster.

name
required
string (ConnectionNamePayload) [ 1 .. 64 ] characters

The name of a connection. A connection name can only contain ASCII letters, numbers, and the following characters: ., _, -. The maximum length of a connection name is 64.

required
object (ConnectionSecretsRequiredPayload)

Base type for connection secrets.

topicName
required
string

Name of the Kafka topic in Confluent.

type
required
string (ConnectionTypePayload)
description
string (ConnectionDescriptionPayload)

Description for a connection.

Responses

Request samples

Content type
application/json
Example
{
  • "name": "example-connection",
  • "type": "confluent",
  • "description": "string",
  • "bootstrapServers": "pkc-xxxxx.us-east-1.aws.confluent.cloud:9092",
  • "topicName": "example-topic",
  • "secrets": {
    }
}

Response samples

Content type
application/json
Example
{
  • "modifiedByUser": {
    },
  • "modifiedOnTimestamp": "2019-08-24T14:15:22Z",
  • "name": "example-connection",
  • "submittedByUser": {
    },
  • "submittedOnTimestamp": "2019-08-24T14:15:22Z",
  • "type": "confluent",
  • "description": "string",
  • "bootstrapServers": "pkc-xxxxx.us-east-1.aws.confluent.cloud:9092",
  • "topicName": "example-topic",
  • "secrets": {
    }
}

Get connection details

Returns information for a single connection.

Authorizations:
apiKeyAuthbearerAuth
path Parameters
name
required
string (ConnectionNamePayload) [ 1 .. 64 ] characters
Example: example-connection

The name of the connection to get.

Responses

Response samples

Content type
application/json
Example
{
  • "modifiedByUser": {
    },
  • "modifiedOnTimestamp": "2019-08-24T14:15:22Z",
  • "name": "example-connection",
  • "submittedByUser": {
    },
  • "submittedOnTimestamp": "2019-08-24T14:15:22Z",
  • "type": "confluent",
  • "description": "string",
  • "bootstrapServers": "pkc-xxxxx.us-east-1.aws.confluent.cloud:9092",
  • "topicName": "example-topic",
  • "secrets": {
    }
}

Update a connection

Updates details for a specific connection.

Authorizations:
apiKeyAuthbearerAuth
path Parameters
name
required
string

The name of the connection to update.

Request Body schema: application/json

The updated connection spec.

type
required
string (ConnectionTypePayload)
bootstrapServers
string

List of host and port pairs used for establishing the initial connection to the Kafka cluster.

description
string (ConnectionDescriptionPayload)

Description for a connection.

object (ConnectionSecretsOptionalPayload)

Base type for updating connection secrets.

topicName
string

Name of the Kafka topic in Confluent.

Responses

Request samples

Content type
application/json
Example
{
  • "type": "confluent",
  • "description": "string",
  • "bootstrapServers": "pkc-xxxxx.us-east-1.aws.confluent.cloud:9092",
  • "topicName": "example-topic",
  • "secrets": {
    }
}

Response samples

Content type
application/json
Example
{
  • "modifiedByUser": {
    },
  • "modifiedOnTimestamp": "2019-08-24T14:15:22Z",
  • "name": "example-connection",
  • "submittedByUser": {
    },
  • "submittedOnTimestamp": "2019-08-24T14:15:22Z",
  • "type": "confluent",
  • "description": "string",
  • "bootstrapServers": "pkc-xxxxx.us-east-1.aws.confluent.cloud:9092",
  • "topicName": "example-topic",
  • "secrets": {
    }
}

Delete a connection

Deletes an existing connection.

Authorizations:
apiKeyAuthbearerAuth
path Parameters
name
required
string

The name the connection to delete.

Responses

Response samples

Content type
application/json
{ }

Test a connection

Checks whether an existing connection is valid.

Authorizations:
apiKeyAuthbearerAuth
path Parameters
name
required
string

The name of the connection to test.

Responses

Response samples

Content type
application/json
{ }

Get meta information from connection

Returns meta information related to connections.

Authorizations:
apiKeyAuthbearerAuth

Responses

Response samples

Content type
application/json
{
  • "externalId": "123e4567-e89b-12d3-a456-426614174000",
  • "implyArn": "arn:aws:iam::123456789012:role/imply-cluster-1234567"
}

Jobs v2 (deprecated)

Manage jobs in Imply Polaris.

List all jobs

Returns all jobs.

Authorizations:
apiKeyAuthbearerAuth
query Parameters
connectionName
Array of strings (ConnectionNamePayload) [ items [ 1 .. 64 ] characters ]
Example: connectionName=example-connection

The connection to filter jobs by.

executionStatus
Array of strings (JobExecutionStatusV2)
Items Enum: "pending" "running" "completed" "idle" "canceled" "failed" "unknown"

Filters results with a certain execution status. For example, to list running jobs only, use the address /v2/jobs?executionStatus=running.

healthStatus
Array of strings (JobHealthStatusV2)
Items Enum: "ok" "warn" "error"

Filters results with a certain health status. For example, to list jobs with warnings or errors, use the address /v2/jobs?healthStatus=warn&healthStatus=error.

paginationLimit
integer [ 1 .. 1000 ]
Default: 100

A result limit for pagination.

paginationOffset
integer >= 0
Default: 0

A result offset for pagination.

searchString
string

Filter results to those jobs where the given string is contained in one of the job's fields.

sortColumn
string (JobSortSearchColumn)
Default: "last_updated_date_time"
Enum: "created_by_user_name" "created_date_time" "execution_status" "last_updated_date_time" "source_type" "source_connection_name" "source_table_name" "target_table_name" "type"

The column to sort job results by.

sortOrder
string (SortOrder)
Default: "DESC"
Enum: "ASC" "DESC"
Example: sortOrder=ASC

The order to sort job results by.

targetTableId
string <uuid> (TableId)

Filters results for jobs associated with a particular target table id.

targetTableName
string (TableName) [ 1 .. 255 ] characters
Example: targetTableName=example-table

Filters results for jobs associated with a particular target table name.

type
Array of strings (JobTypeV2)
Items Enum: "batch" "delete_data" "drop_table" "streaming" "sql"
Example: type=batch

Filters results with a certain job type. For example, to select batch and streaming jobs, use the address /v2/jobs?type=batch&type=streaming.

Responses

Response samples

Content type
application/json
{
  • "values": [
    ],
  • "moreResultsAvailable": true
}

Create a job

Creates a job.

Authorizations:
apiKeyAuthbearerAuth
Request Body schema: application/json

A job to be created.

required
object (JobSourceV2)

The source input of the job.

required
object (JobTargetV2)

The target output of the job.

type
required
string (JobTypeV2)
Default: "sql"

The type of job:

  • batch - A batch ingestion job.
  • delete_data - A job to delete data within a table.
  • drop_table - A job to delete a table and all of its data.
  • streaming - A streaming ingestion job.
  • sql - A SQL-based ingestion job.

For information about ingestion jobs, see Create an ingestion job and Ingest using SQL.

desiredExecutionStatus
string (DesiredJobExecutionStatusV2)
Default: "running"
Enum: "running" "canceled"

The desired execution status of the job. This field only applies to updating an ingestion job and is ignored when creating jobs. You cannot update a deletion job. The default desired execution status is running for jobs that should run to completion. Otherwise, specify the value canceled to cancel an ingestion job. If an unsupported value is supplied, Polaris raises a 400 Bad Request error.

filterExpression
string

SQL expression to use for filtering the source data.

ingestionMode
string (IngestionModeV2)
Default: "append"
Enum: "append" "replace"

The mode of ingestion:

  • append - Append newly ingested data to the existing data. To ingest a certain time range of source data, apply a filter expression to filter by a given time interval.
  • replace - Overwrite existing data corresponding to the time interval supplied in target.intervals. The value in target.intervals determines what table data is replaced. To restrict the time interval of what data Polaris ingests, use filter expressions.
Array of objects (InputToTableSchemaMappingsV2)

Relationship between the input fields and the output columns of a table's schema. Define the Polaris column name in columnName, and specify how the column is defined with respect to the input fields in expression. expression must be a valid SQL expression.

For batch ingestion jobs, you must include a mapping for each field. Polaris ignores unmapped input fields. Streaming ingestion jobs may apply schema auto-discovery for Polaris to discover input fields and map them directly to table dimensions. Always specify a mapping for __time and when you want to transform an input field.

For more information, see Map and transform data with input expressions and Schema auto-discovery on ingestion jobs.

maxParseExceptions
integer
Default: 2147483647

Configuration parameter that specifies the maximum number of parse exceptions that a task can tolerate before failing.

Responses

Request samples

Content type
application/json
Example
{
  • "type": "batch",
  • "desiredExecutionStatus": "running",
  • "source": {
    },
  • "target": {
    },
  • "filterExpression": "\"dimension\" > 0",
  • "ingestionMode": "append",
  • "mappings": [
    ],
  • "maxParseExceptions": 2147483647
}

Response samples

Content type
application/json
Example
{
  • "createdBy": {
    },
  • "createdTimestamp": "2021-06-06T00:00:00.000Z",
  • "desiredExecutionStatus": "running",
  • "executionStatus": "pending",
  • "health": {
    },
  • "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  • "lastModifiedBy": {
    },
  • "lastUpdatedTimestamp": "2021-06-06T00:00:00.000Z",
  • "spec": {
    },
  • "target": {
    },
  • "type": "batch",
  • "completedTimestamp": "2021-06-06T00:00:00.000Z",
  • "startedTimestamp": "2021-06-06T00:00:00.000Z",
  • "source": {
    },
  • "filterExpression": "\"dimension\" > 0",
  • "ingestionMode": "append",
  • "mappings": [
    ],
  • "maxParseExceptions": 2147483647,
  • "query": "string"
}

Get job details

Returns information for a single job.

Authorizations:
apiKeyAuthbearerAuth
path Parameters
jobId
required
string <uuid> (JobIdPayload)

The server-generated ID of the job for which to get the metadata.

Responses

Response samples

Content type
application/json
Example
{
  • "createdBy": {
    },
  • "createdTimestamp": "2021-06-06T00:00:00.000Z",
  • "desiredExecutionStatus": "running",
  • "executionStatus": "pending",
  • "health": {
    },
  • "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  • "lastModifiedBy": {
    },
  • "lastUpdatedTimestamp": "2021-06-06T00:00:00.000Z",
  • "spec": {
    },
  • "target": {
    },
  • "type": "batch",
  • "completedTimestamp": "2021-06-06T00:00:00.000Z",
  • "startedTimestamp": "2021-06-06T00:00:00.000Z",
  • "source": {
    },
  • "filterExpression": "\"dimension\" > 0",
  • "ingestionMode": "append",
  • "mappings": [
    ],
  • "maxParseExceptions": 2147483647,
  • "query": "string"
}

Update a job

Updates a job.

You can update an ingestion job to cancel its execution. Polaris does not support other updates to ingestion jobs. Deletion jobs cannot be updated.

Authorizations:
apiKeyAuthbearerAuth
path Parameters
jobId
required
string <uuid> (JobIdPayload)

The server-generated ID of the job for which to get the metadata.

Request Body schema: application/json

The job to be updated.

desiredExecutionStatus
required
string (DesiredJobExecutionStatusV2)
Default: "running"
Enum: "running" "canceled"

The desired execution status of the job. This field only applies to updating an ingestion job and is ignored when creating jobs. You cannot update a deletion job. The default desired execution status is running for jobs that should run to completion. Otherwise, specify the value canceled to cancel an ingestion job. If an unsupported value is supplied, Polaris raises a 400 Bad Request error.

Responses

Request samples

Content type
application/json
{
  • "desiredExecutionStatus": "running"
}

Response samples

Content type
application/json
Example
{
  • "createdBy": {
    },
  • "createdTimestamp": "2021-06-06T00:00:00.000Z",
  • "desiredExecutionStatus": "running",
  • "executionStatus": "pending",
  • "health": {
    },
  • "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  • "lastModifiedBy": {
    },
  • "lastUpdatedTimestamp": "2021-06-06T00:00:00.000Z",
  • "spec": {
    },
  • "target": {
    },
  • "type": "batch",
  • "completedTimestamp": "2021-06-06T00:00:00.000Z",
  • "startedTimestamp": "2021-06-06T00:00:00.000Z",
  • "source": {
    },
  • "filterExpression": "\"dimension\" > 0",
  • "ingestionMode": "append",
  • "mappings": [
    ],
  • "maxParseExceptions": 2147483647,
  • "query": "string"
}

Get job logs

Returns logs for a job.

Authorizations:
apiKeyAuthbearerAuth
path Parameters
jobId
required
string <uuid> (JobIdPayload)

The server-generated ID of the job for which to get the metadata.

Responses

Response samples

Content type
application/json
{
  • "logs": [
    ]
}

Get job metrics

Returns detailed metrics about a job.

Authorizations:
apiKeyAuthbearerAuth
path Parameters
jobId
required
string <uuid> (JobIdPayload)

The server-generated ID of the job for which to get the metadata.

Responses

Response samples

Content type
application/json
{
  • "totals": {
    }
}

Reset job to apply streaming ingestion offset

Resets a streaming ingestion job. This resets the point from which Polaris ingests streaming data to readFromPoint set in current streaming ingestion jobs. Only applies when there was a previous streaming ingestion job that used the same topic or stream name for the same target table. For more information, see Reset streaming job offset.

Authorizations:
apiKeyAuthbearerAuth
path Parameters
jobId
required
string <uuid> (JobIdPayload)

The server-generated ID of the job

Responses

Response samples

Content type
application/json
{ }

Tables v2 (deprecated)

Manage tables and their schemas.

List all tables

Returns all available tables.

Authorizations:
apiKeyAuthbearerAuth

Responses

Response samples

Content type
application/json
{
  • "values": [
    ]
}

Create a table

Creates a table.

Authorizations:
apiKeyAuthbearerAuth
Request Body schema: application/json

A table to be created.

name
required
string (TableName) [ 1 .. 255 ] characters

The unique, immutable name of the table.

type
required
string (TableType)

The type of table. Once set, you cannot change a table's type. The following table types are supported:

  • detail - A detail table is a standard table that stores each ingested record as a row in the table.
  • aggregate - An aggregate table combines multiple rows based on dimension values and the table's timeResolution, resulting in a condensed data set for improved query performance. See Introduction to data rollup for more details.
version
required
integer <int64> (TableVersion)
Default: 0

The version number of the table. Newly created tables are always initialized at version 0. Do not provide the version number when creating a table. When updating a table, provide the current table version number. If the version number in the request is not the current version, Polaris raises a 409 Conflict error. In that case, retrieve the latest version and retry the request to avoid concurrently conflicting changes.

clusteringColumns
Array of strings or null (TableClusteringColumns)

Table columns to use for clustering. Polaris sorts table columns within the partition based on their order in this array.

For aggregate tables, only dimensions (not measures) are supported for clustering. If a nonexistent column is specified, Polaris raises a 400 Bad Request error.

For more information, see Clustering.

description
string or null (TableDescription)

The table's description, limited to 1000 characters.

partitioningGranularity
string (TablePartitioningGranularity)
Default: "day"
Enum: "hour" "day" "week" "month" "year"

The time partitioning of the table. The week time period is deprecated. You can't create a new table with the time partitioning granularity set to week.

Array of objects (AggregateTableColumn)

The table's schema. The order of the elements determines the order of the columns in the Polaris UI and in SELECT * SQL queries.

schemaMode
string (TableSchemaMode)
Enum: "flexible" "strict"

The table's schema enforcement mode.

object (TableStoragePolicy)

Policy specifying which data to retain and which data to drop from the table.

timeResolution
string (TimeResolution)
Default: "millisecond"
Enum: "millisecond" "second" "minute" "fifteen_minute" "thirty_minute" "hour" "day" "week" "month" "quarter" "year"

The table's rollup granularity. See Introduction to data rollup for more details.

Responses

Request samples

Content type
application/json
Example
{
  • "name": "example-table",
  • "type": "aggregate",
  • "version": 0,
  • "clusteringColumns": [
    ],
  • "description": "Description of example-table",
  • "partitioningGranularity": "day",
  • "storagePolicy": {
    },
  • "schemaMode": "flexible",
  • "schema": [
    ],
  • "timeResolution": "millisecond"
}

Response samples

Content type
application/json
Example
{
  • "name": "example-table",
  • "type": "aggregate",
  • "version": 0,
  • "availability": "available",
  • "clusteringColumns": [
    ],
  • "createdByUser": {
    },
  • "createdOnTimestamp": "2019-08-24T14:15:22Z",
  • "description": "Description of example-table",
  • "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  • "modifiedByUser": {
    },
  • "modifiedOnTimestamp": "2021-06-06T00:00:00.000Z",
  • "partitioningGranularity": "day",
  • "queryableSchema": [
    ],
  • "storagePolicy": {
    },
  • "schemaMode": "flexible",
  • "segmentCompactedBytes": 12345678,
  • "segmentTotalBytes": 12345678,
  • "totalDataSizeBytes": 12345678,
  • "totalRows": 1234,
  • "schema": [
    ],
  • "timeResolution": "millisecond"
}

Get table details

Returns information for a single table.

Authorizations:
apiKeyAuthbearerAuth
path Parameters
tableRef
required
string (TableName) [ 1 .. 255 ] characters
Example: example-table

Name of the table for which to get information.

Responses

Response samples

Content type
application/json
Example
{
  • "name": "example-table",
  • "type": "aggregate",
  • "version": 0,
  • "availability": "available",
  • "clusteringColumns": [
    ],
  • "createdByUser": {
    },
  • "createdOnTimestamp": "2019-08-24T14:15:22Z",
  • "description": "Description of example-table",
  • "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  • "modifiedByUser": {
    },
  • "modifiedOnTimestamp": "2021-06-06T00:00:00.000Z",
  • "partitioningGranularity": "day",
  • "queryableSchema": [
    ],
  • "storagePolicy": {
    },
  • "schemaMode": "flexible",
  • "segmentCompactedBytes": 12345678,
  • "segmentTotalBytes": 12345678,
  • "totalDataSizeBytes": 12345678,
  • "totalRows": 1234,
  • "schema": [
    ],
  • "timeResolution": "millisecond"
}

Update a table

Updates a table.

You can add columns to a table's schema at any point. You can only update or remove columns from a table's schema when the table is empty and has no active ingestion jobs.

Authorizations:
apiKeyAuthbearerAuth
path Parameters
tableRef
required
string

Name of the table to update.

Request Body schema: application/json

The table to be updated.

name
required
string (TableName) [ 1 .. 255 ] characters

The unique, immutable name of the table.

type
required
string (TableType)

The type of table. Once set, you cannot change a table's type. The following table types are supported:

  • detail - A detail table is a standard table that stores each ingested record as a row in the table.
  • aggregate - An aggregate table combines multiple rows based on dimension values and the table's timeResolution, resulting in a condensed data set for improved query performance. See Introduction to data rollup for more details.
version
required
integer <int64> (TableVersion)
Default: 0

The version number of the table. Newly created tables are always initialized at version 0. Do not provide the version number when creating a table. When updating a table, provide the current table version number. If the version number in the request is not the current version, Polaris raises a 409 Conflict error. In that case, retrieve the latest version and retry the request to avoid concurrently conflicting changes.

clusteringColumns
Array of strings or null (TableClusteringColumns)

Table columns to use for clustering. Polaris sorts table columns within the partition based on their order in this array.

For aggregate tables, only dimensions (not measures) are supported for clustering. If a nonexistent column is specified, Polaris raises a 400 Bad Request error.

For more information, see Clustering.

description
string or null (TableDescription)

The table's description, limited to 1000 characters.

partitioningGranularity
string (TablePartitioningGranularity)
Default: "day"
Enum: "hour" "day" "week" "month" "year"

The time partitioning of the table. The week time period is deprecated. You can't create a new table with the time partitioning granularity set to week.

Array of objects (AggregateTableColumn)

The table's schema. The order of the elements determines the order of the columns in the Polaris UI and in SELECT * SQL queries.

schemaMode
string (TableSchemaMode)
Enum: "flexible" "strict"

The table's schema enforcement mode.

object (TableStoragePolicy)

Policy specifying which data to retain and which data to drop from the table.

timeResolution
string (TimeResolution)
Default: "millisecond"
Enum: "millisecond" "second" "minute" "fifteen_minute" "thirty_minute" "hour" "day" "week" "month" "quarter" "year"

The table's rollup granularity. See Introduction to data rollup for more details.

Responses

Request samples

Content type
application/json
Example
{
  • "name": "example-table",
  • "type": "aggregate",
  • "version": 0,
  • "clusteringColumns": [
    ],
  • "description": "Description of example-table",
  • "partitioningGranularity": "day",
  • "storagePolicy": {
    },
  • "schemaMode": "flexible",
  • "schema": [
    ],
  • "timeResolution": "millisecond"
}

Response samples

Content type
application/json
Example
{
  • "name": "example-table",
  • "type": "aggregate",
  • "version": 0,
  • "availability": "available",
  • "clusteringColumns": [
    ],
  • "createdByUser": {
    },
  • "createdOnTimestamp": "2019-08-24T14:15:22Z",
  • "description": "Description of example-table",
  • "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  • "modifiedByUser": {
    },
  • "modifiedOnTimestamp": "2021-06-06T00:00:00.000Z",
  • "partitioningGranularity": "day",
  • "queryableSchema": [
    ],
  • "storagePolicy": {
    },
  • "schemaMode": "flexible",
  • "segmentCompactedBytes": 12345678,
  • "segmentTotalBytes": 12345678,
  • "totalDataSizeBytes": 12345678,
  • "totalRows": 1234,
  • "schema": [
    ],
  • "timeResolution": "millisecond"
}