Skip to main content

View and manage jobs by API

After you create an ingestion job for Imply Polaris, you can get its status, view related metrics, and stop its execution. This topic walks you through the process of viewing job-related details and canceling a job.

For details on starting ingestion jobs, see Create an ingestion job by API.

Prerequisites​

This topic assumes that you have an API key with the ManageIngestionJobs permission. In the examples below, the key value is stored in the variable named POLARIS_API_KEY. For information about how to obtain an API key and assign permissions, see API key authentication. For more information on permissions, visit Permissions reference.

Get all jobs​

To retrieve information about all jobs, send a GET request to the Jobs v1 API:

/v1/projects/PROJECT_ID/jobs

Replace PROJECT_ID with your Polaris project ID.

Filter jobs on a single attribute​

Use query parameters to only return jobs that have certain attributes. This allows for more precise and relevant results. To pass in a query parameter, concatenate the /jobs endpoint URL with a ? followed by the key-value pair of the parameter.

To filter jobs on a single attribute, send a GET request to /v1/projects/PROJECT_ID/jobs? with the parameter following the ?. For example, the following call returns only those jobs whose healthStatus property has a value of error.

/v1/projects/PROJECT_ID/jobs?healthStatus=error

Filter jobs on multiple attributes​

To create more complex filters that use multiple query parameters, delimit each parameter with a &. Issue a GET request to the following endpoint to return all jobs that match both healthStatus=ok and type=batch.

/v1/projects/PROJECT_ID/jobs?healthStatus=ok&type=batch

You can add multiple instances of the same query parameter to return jobs that match any of the specified values. Issue a GET request to the following endpoint to return jobs with the healthStatus=ok or the healthStatus=error attribute. Note that if there are multiple instances of the same query parameter, a job only needs to match one of them.

/v1/projects/PROJECT_ID/jobs?healthStatus=ok&healthStatus=error

Sample request​

The following example shows how to return jobs with multiple query parameters. The example requests batch ingestion jobs that have a health status of either ok or error.

curl --location --request GET "https://ORGANIZATION_NAME.REGION.CLOUD_PROVIDER.api.imply.io/v1/projects/PROJECT_ID/jobs?type=batch&healthStatus=ok&healthStatus=error' \
--header 'Accept: application/json' \
--header "Authorization: Basic $POLARIS_API_KEY"

Sample response​

A successful request returns a 200 OK response and the details of each job that matches the filter:

{
    "values": [
        {
            "source": {
                "data": "{example-data}",
                "type": "inline"
            },
            "createdBy": {
                "username": "api-key-pok_XXXXX...XXXXXX",
                "userId": "b6340b70-3f30-XXXX-XXXX-XXXXXXXXXXXX"
            },
            "createdTimestamp": "2024-08-01TXX:XX:XX.XXXXXXX",
            "desiredExecutionStatus": "running",
            "executionStatus": "completed",
            "health": {
                "status": "ok"
            },
            "id": "0191104b-7e8e-XXXX-XXXX-XXXXXXXXXXXX",
            "lastModifiedBy": {
                "username": "api-key-pok_XXXXX...XXXXXX",
                "userId": "b6340b70-3f30-XXXX-XXXX-XXXXXXXXXXXX"
            },
            "lastUpdatedTimestamp": "2024-08-01TXX:XX:XX.XXXXXXX",
            "target": {
                "tableName": "example-table",
                "type": "table",
                "intervals": null
            },
            "type": "batch",
            "completedTimestamp": "2024-08-01TXX:XX:XX.XXXXXXX",
            "startedTimestamp": "2024-08-01TXX:XX:XX.XXXXXXX",
        },
         ...
      }
    ]
}

Get job status​

To get the metadata of a job, submit a GET request to the following endpoint. Replace PROJECT_ID with your Polaris project ID and JOB_ID with the ID of your job.

/v1/projects/PROJECT_ID/jobs/JOB_ID

To get just the status of a job, send a GET request to the following endpoint:

/v1/projects/PROJECT_ID/jobs/JOB_ID/status

Sample request​

The following example shows how to get the status of a job that has job ID efb35e3e-406e-4127-ad2e-280fede4f431:

curl --location --request GET "https://ORGANIZATION_NAME.REGION.CLOUD_PROVIDER.api.imply.io/v1/projects/PROJECT_ID/jobs/efb35e3e-406e-4127-ad2e-280fede4f431/status" \
--header "Authorization: Basic $POLARIS_API_KEY" \
--data-raw ''

Sample response​

The following example shows a successful response for a job's progress:

{
  "executionStatus": "idle"
}

Update job status​

Use the job status API to change the execution status of a job.

Note that you can also update a job's status by sending a PUT request to the jobs endpoint. However, it's recommended you send a POST request to the job status endpoint, as described in this section.

Stop a job​

To stop a job, issue a POST request to the job status API, and in your request body set desiredExecutionStatus to canceled.

{
  "desiredExecutionStatus": "canceled"
}

For details on stopping jobs, see Manage jobs.

Pause or resume a streaming job​

You can pause and resume streaming ingestion jobs. In your request to the job status API, send the following request body to pause or resume a job, respectively:

Pause a job:

{
  "desiredExecutionStatus": "suspended"
}

Resume a paused job:

{
  "desiredExecutionStatus": "running"
}

Sample request​

The following example shows how to stop a job with the job ID efb35e3e-406e-4127-ad2e-280fede4f431:

curl --location --request PUT "https://ORGANIZATION_NAME.REGION.CLOUD_PROVIDER.api.imply.io/v1/projects/PROJECT_ID/jobs/efb35e3e-406e-4127-ad2e-280fede4f431/status" \
--header "Authorization: Basic $POLARIS_API_KEY" \
--header "Content-Type: application/json" \
--data-raw '{
    "desiredExecutionStatus": "canceled"
}'

Sample response​

When you successfully stop a job, the Jobs v1 API returns the 200 OK status code and the details of the job.

View job metrics​

Send a GET request to /v1/projects/PROJECT_ID/jobs/JOB_ID/metrics to view the metrics for a job.

The following example shows metrics for an ingestion job in which two rows could not be parsed. By default, Polaris continues to ingest data when it encounters rows it cannot parse. You can specify a threshold for parsing exceptions for a job. The job fails if the number of parsing exceptions exceeds the threshold. Set this limit in maxParseExceptions when creating an ingestion job. You can view parse exception messages in the job logs.

You can also use the UI to monitor the health of your ingestion job. For more information, see Monitor streaming jobs.

Sample request​

The following example shows how to view metrics for an ingestion job with the job ID db6a3110-d6c3-4a63-86b2-41d51a65ce11:

curl --location --request GET "https://ORGANIZATION_NAME.REGION.CLOUD_PROVIDER.api.imply.io/v1/projects/PROJECT_ID/jobs/db6a3110-d6c3-4a63-86b2-41d51a65ce11/metrics" \
--header "Authorization: Basic $POLARIS_API_KEY"

Sample response​

The following example shows a successful response for metrics of an ingestion job:

{
    "totals": {
        "numRowsProcessed": 99999,
        "numRowsProcessedWithWarning": 0,
        "numRowsSkippedByFilter": 0,
        "numRowsSkippedByError": 2
    }
}

Learn more​

See the following topics for more information: