Delete data by API

info

Project-less regional API resources have been deprecated and will be removed by the end of September 2024.

You must include the project ID in the URL for all regional API calls in projects created after September 29, 2023. For example: https://ORGANIZATION_NAME.REGION.CLOUD_PROVIDER.api.imply.io/v1/projects/PROJECT_ID

Projects created before September 29, 2023 can continue to use project-less URLs until the end of September 2024. We strongly recommend updating your regional API calls to include the project ID prior to September 2024. See the API migration guide for more information.

You can use the Imply Polaris API to delete data in a table, whether you want to remove all data or a subset of data that falls within a given time interval. For information on deleting data using the UI, see Delete data.

Delete behavior

By default, requests to delete data using the Polaris API permanently deletes the data. Permanently deleted data reduces your data usage, which can help you stay within your project size.

If you want the ability to restore the data within a 30 day grace period, include "softDelete": True in your requests. When you include "softDelete": True in drop_table and delete_data job requests, Polaris soft deletes the data instead of permanently deleting it. That means the data can be restored.

Note that when you replace data, Polaris soft deletes the old version of the data as part of the job.

For information about restoring data, see either Restore data by API or Restore or permanently delete data for the UI.

Prerequisites

This topic assumes that you have the following:

A table containing data.
An API key with the ManageIngestionJobs and ManageTables permissions. In the examples below, the key value is stored in the variable named POLARIS_API_KEY. To obtain an API key and assign permissions, see API key authentication. For more information on permissions, visit Permissions reference.

Soft delete data

Data deletion is a job in Polaris in which the job type is delete_data. The inclusion of "softDelete": True in the API calls means that the data gets soft deleted, which is not the default behavior. Soft deleted data is restorable if it's within the 30 day grace period and hasn't been permanently deleted.

Soft delete data by time interval

To remove a subset of data based on the time interval, provide the time interval in ISO 8601 format in target.intervals.

Sample request

The following example shows a delete_data job to soft delete data within a given time interval. See the Jobs v1 API documentation for more information.

cURL
Python

curl --location --request POST "https://ORGANIZATION_NAME.REGION.CLOUD_PROVIDER.api.imply.io/v1/projects/PROJECT_ID/jobs" \
--header "Authorization: Basic $POLARIS_API_KEY" \
--header "Content-Type: application/json" \
--data '{
    "type": "delete_data",
    "softDelete": true,
    "target": {
        "type": "table",
        "tableName": "demo_table",
        "intervals": ["2022-07-01/2022-08-01"]
    }
}'

import os
import requests
import json

url = "https://ORGANIZATION_NAME.REGION.CLOUD_PROVIDER.api.imply.io/v1/projects/PROJECT_ID/jobs"

apikey = os.getenv("POLARIS_API_KEY")

payload = json.dumps({
  "type": "delete_data",
  "softDelete": True,
  "target": {
    "type": "table",
    "tableName": "demo_table",
    "intervals": [
      "2022-07-01/2022-08-01"
    ]
  }
})
headers = {
  'Authorization': f'Basic {apikey}',
  'Content-Type': 'application/json'
}

response = requests.request("POST", url, headers=headers, data=payload)

print(response.text)

Sample response

The following example shows a successful response:

Click to view the response

{
    "deleteAll": false,
    "createdBy": {
        "username": "api-key-pok_vipgj...bjjvyo",
        "userId": "a52cacf6-3ddc-48e5-8675-xxxxxxxxxxxx"
    },
    "createdTimestamp": "2023-08-10T00:14:21.188501674Z",
    "desiredExecutionStatus": "running",
    "executionStatus": "pending",
    "health": {
        "status": "ok"
    },
    "id": "0189dccb-5804-7bf9-bbbb-6ac085b71b44",
    "lastModifiedBy": {
        "username": "api-key-pok_vipgj...bjjvyo",
        "userId": "a52cacf6-3ddc-48e5-8675-xxxxxxxxxxxx"
    },
    "lastUpdatedTimestamp": "2023-08-10T00:14:21.188501674Z",
    "spec": {
        "target": {
            "tableName": "demo_table",
            "type": "table",
            "intervals": [
                "2022-07-01/2022-08-01"
            ]
        },
        "deleteAll": false,
        "type": "delete_data",
        "desiredExecutionStatus": "running"
    },
    "target": {
        "tableName": "demo_table",
        "type": "table",
        "intervals": []
    },
    "type": "delete_data",
    "completedTimestamp": null,
    "startedTimestamp": null
}

Soft delete all data

To remove all data within a table, submit a delete_data type job and set the deleteAll property to true in addition to "softDelete": True .

Sample request

The following example shows a delete_data job to soft delete all the data in a table. See the Jobs v1 API documentation for more information.

cURL
Python

curl --location --request POST "https://ORGANIZATION_NAME.REGION.CLOUD_PROVIDER.api.imply.io/v1/projects/PROJECT_ID/jobs" \
--header "Authorization: Basic $POLARIS_API_KEY" \
--header "Content-Type: application/json" \
--data '{
    "type": "delete_data",
    "softDelete": true,
    "target": {
        "type": "table",
        "tableName": "demo_table"
    },
    "deleteAll": true
}'

import os
import requests
import json

url = "https://ORGANIZATION_NAME.REGION.CLOUD_PROVIDER.api.imply.io/v1/projects/PROJECT_ID/jobs"

apikey = os.getenv("POLARIS_API_KEY")

payload = json.dumps({
  "type": "delete_data",
  "softDelete": True,
  "target": {
    "type": "table",
    "tableName": "demo_table"
  },
  "deleteAll": True
})
headers = {
  'Authorization': f'Basic {apikey}',
  'Content-Type': 'application/json'
}

response = requests.request("POST", url, headers=headers, data=payload)

print(response.text)

Sample response

The following example shows a successful response:

Click to view the response

{
    "deleteAll": true,
    "createdBy": {
        "username": "api-key-pok_vipgj...bjjvyo",
        "userId": "a52cacf6-3ddc-48e5-8675-xxxxxxxxxxxx"
    },
    "createdTimestamp": "2023-08-10T00:23:40.518838888Z",
    "desiredExecutionStatus": "running",
    "executionStatus": "pending",
    "health": {
        "status": "ok"
    },
    "id": "0189dcd3-e0e6-7d8c-9f1d-0a338a4b443b",
    "lastModifiedBy": {
        "username": "api-key-pok_vipgj...bjjvyo",
        "userId": "a52cacf6-3ddc-48e5-8675-xxxxxxxxxxxx"
    },
    "lastUpdatedTimestamp": "2023-08-10T00:23:40.518838888Z",
    "spec": {
        "target": {
            "tableName": "demo_table",
            "type": "table",
            "intervals": []
        },
        "deleteAll": true,
        "type": "delete_data",
        "desiredExecutionStatus": "running"
    },
    "target": {
        "tableName": "demo_table",
        "type": "table",
        "intervals": []
    },
    "type": "delete_data",
    "completedTimestamp": null,
    "startedTimestamp": null
}

Drop a table and soft delete its data

To delete a table and all of its data, create a job with its type set to drop_table.

Sample request

The following example shows a drop_table job to soft delete a table and its data. See the Jobs v1 API documentation for more information.

cURL
Python

curl --location --request POST "https://ORGANIZATION_NAME.REGION.CLOUD_PROVIDER.api.imply.io/v1/projects/PROJECT_ID/jobs" \
--header "Authorization: Basic $POLARIS_API_KEY" \
--header "Content-Type: application/json" \
--data '{
    "type": "drop_table",
    "softDelete": true,
    "target": {
        "type": "table",
        "tableName": "Koalas"
    }
}'

import os
import requests
import json

url = "https://ORGANIZATION_NAME.REGION.CLOUD_PROVIDER.api.imply.io/v1/projects/PROJECT_ID/jobs"

apikey = os.getenv("POLARIS_API_KEY")

payload = json.dumps({
  "type": "drop_table",
  "softDelete": True,
  "target": {
    "type": "table",
    "tableName": "Koalas"
  }
})
headers = {
  'Authorization': f'Basic {apikey}',
  'Content-Type': 'application/json'
}

response = requests.request("POST", url, headers=headers, data=payload)

print(response.text)

Sample response

The following example shows a successful response:

Click to view the response

{
    "createdBy": {
        "username": "api-key-pok_vipgj...bjjvyo",
        "userId": "a52cacf6-3ddc-48e5-8675-xxxxxxxxxxxx"
    },
    "createdTimestamp": "2023-08-10T00:25:22.094458188Z",
    "desiredExecutionStatus": "running",
    "executionStatus": "pending",
    "health": {
        "status": "ok"
    },
    "id": "0189dcd5-6dae-72c1-a04c-8caceba85a32",
    "lastModifiedBy": {
        "username": "api-key-pok_vipgj...bjjvyo",
        "userId": "a52cacf6-3ddc-48e5-8675-xxxxxxxxxxxx"
    },
    "lastUpdatedTimestamp": "2023-08-10T00:25:22.094458188Z",
    "spec": {
        "target": {
            "tableName": "Koalas",
            "type": "table",
            "intervals": []
        },
        "type": "drop_table",
        "desiredExecutionStatus": "running"
    },
    "target": {
        "tableName": "Koalas",
        "type": "table",
        "intervals": []
    },
    "type": "drop_table",
    "completedTimestamp": null,
    "startedTimestamp": null
}

Permanently delete data

The API calls for a soft delete and a permanent deletion are identical except permanent deletion omits "softDelete": True. Permanent deletion of data is the default behavior when you delete data through the API.

Permanently delete data by time interval

Data deletion is a job in Polaris, in which the job type is delete_data. To remove a subset of data based on the time interval, provide the time intervals in ISO 8601 format in target.intervals.

Note that this property is also used for replacing data in a table. The time interval identifies the data that already exists in the table based on values in the __time column.

Sample request

The following example shows a delete_data job to delete data within a given time interval. See the Jobs v1 API documentation for more information.

cURL
Python

curl --location --request POST "https://ORGANIZATION_NAME.REGION.CLOUD_PROVIDER.api.imply.io/v1/projects/PROJECT_ID/jobs" \
--user ${POLARIS_API_KEY}: \
--header 'Content-Type: application/json' \
--data '{
    "type": "delete_data",
    "target": {
        "type": "table",
        "tableName": "demo_table",
        "intervals": ["2022-07-01/2022-08-01"]
    }
}'

import os
import requests
import json

url = "https://ORGANIZATION_NAME.REGION.CLOUD_PROVIDER.api.imply.io/v1/projects/PROJECT_ID/jobs"

apikey = os.getenv("POLARIS_API_KEY")

payload = json.dumps({
  "type": "delete_data",
  "target": {
    "type": "table",
    "tableName": "demo_table",
    "intervals": [
      "2022-07-01/2022-08-01"
    ]
  }
})
headers = {
  'Authorization': f'Basic {apikey}',
  'Content-Type': 'application/json'
}

response = requests.request("POST", url, headers=headers, data=payload)

print(response.text)

Sample response

The following example shows a successful response:

Click to view the response

{
    "deleteAll": false,
    "createdBy": {
        "username": "api-key-pok_vipgj...bjjvyo",
        "userId": "a52cacf6-3ddc-48e5-8675-xxxxxxxxxxxx"
    },
    "createdTimestamp": "2023-08-10T00:14:21.188501674Z",
    "desiredExecutionStatus": "running",
    "executionStatus": "pending",
    "health": {
        "status": "ok"
    },
    "id": "0189dccb-5804-7bf9-bbbb-6ac085b71b44",
    "lastModifiedBy": {
        "username": "api-key-pok_vipgj...bjjvyo",
        "userId": "a52cacf6-3ddc-48e5-8675-xxxxxxxxxxxx"
    },
    "lastUpdatedTimestamp": "2023-08-10T00:14:21.188501674Z",
    "spec": {
        "target": {
            "tableName": "demo_table",
            "type": "table",
            "intervals": [
                "2022-07-01/2022-08-01"
            ]
        },
        "deleteAll": false,
        "type": "delete_data",
        "desiredExecutionStatus": "running"
    },
    "target": {
        "tableName": "demo_table",
        "type": "table",
        "intervals": []
    },
    "type": "delete_data",
    "completedTimestamp": null,
    "startedTimestamp": null
}

Permanently delete all data

To permanently remove all data within a table, submit a delete_data type job and set the deleteAll property to true.

Sample request

The following example shows a delete_data job to delete all the data in a table. See the Jobs v1 API documentation for more information.

cURL
Python

curl --location --request POST "https://ORGANIZATION_NAME.REGION.CLOUD_PROVIDER.api.imply.io/v1/projects/PROJECT_ID/jobs" \
--user ${POLARIS_API_KEY}: \
--header 'Content-Type: application/json' \
--data '{
    "type": "delete_data",
    "target": {
        "type": "table",
        "tableName": "demo_table"
    },
    "deleteAll": true
}'

import os
import requests
import json

url = "https://ORGANIZATION_NAME.REGION.CLOUD_PROVIDER.api.imply.io/v1/projects/PROJECT_ID/jobs"

apikey = os.getenv("POLARIS_API_KEY")

payload = json.dumps({
  "type": "delete_data",
  "target": {
    "type": "table",
    "tableName": "demo_table"
  },
  "deleteAll": True
})
headers = {
  'Authorization': f'Basic {apikey}',
  'Content-Type': 'application/json'
}

response = requests.request("POST", url, headers=headers, data=payload)

print(response.text)

Sample response

The following example shows a successful response:

Click to view the response

{
    "deleteAll": true,
    "createdBy": {
        "username": "api-key-pok_vipgj...bjjvyo",
        "userId": "a52cacf6-3ddc-48e5-8675-xxxxxxxxxxxx"
    },
    "createdTimestamp": "2023-08-10T00:23:40.518838888Z",
    "desiredExecutionStatus": "running",
    "executionStatus": "pending",
    "health": {
        "status": "ok"
    },
    "id": "0189dcd3-e0e6-7d8c-9f1d-0a338a4b443b",
    "lastModifiedBy": {
        "username": "api-key-pok_vipgj...bjjvyo",
        "userId": "a52cacf6-3ddc-48e5-8675-xxxxxxxxxxxx"
    },
    "lastUpdatedTimestamp": "2023-08-10T00:23:40.518838888Z",
    "spec": {
        "target": {
            "tableName": "demo_table",
            "type": "table",
            "intervals": []
        },
        "deleteAll": true,
        "type": "delete_data",
        "desiredExecutionStatus": "running"
    },
    "target": {
        "tableName": "demo_table",
        "type": "table",
        "intervals": []
    },
    "type": "delete_data",
    "completedTimestamp": null,
    "startedTimestamp": null
}

Drop a table and permanently delete its data

To delete a table and all of its data, create a job with its type set to drop_table.

Sample request

The following example shows a drop_table job to permanently delete a table and its data. See the Jobs v1 API documentation for more information.

cURL
Python

curl --location --request POST "https://ORGANIZATION_NAME.REGION.CLOUD_PROVIDER.api.imply.io/v1/projects/PROJECT_ID/jobs" \
--user ${POLARIS_API_KEY}: \
--header 'Content-Type: application/json' \
--data '{
    "type": "drop_table",
    "target": {
        "type": "table",
        "tableName": "Koalas"
    }
}'

import os
import requests
import json

url = "https://ORGANIZATION_NAME.REGION.CLOUD_PROVIDER.api.imply.io/v1/projects/PROJECT_ID/jobs"

apikey = os.getenv("POLARIS_API_KEY")

payload = json.dumps({
  "type": "drop_table",
  "target": {
    "type": "table",
    "tableName": "Koalas"
  }
})
headers = {
  'Authorization': f'Basic {apikey}',
  'Content-Type': 'application/json'
}

response = requests.request("POST", url, headers=headers, data=payload)

print(response.text)

Sample response

The following example shows a successful response:

Click to view the response

{
    "createdBy": {
        "username": "api-key-pok_vipgj...bjjvyo",
        "userId": "a52cacf6-3ddc-48e5-8675-xxxxxxxxxxxx"
    },
    "createdTimestamp": "2023-08-10T00:25:22.094458188Z",
    "desiredExecutionStatus": "running",
    "executionStatus": "pending",
    "health": {
        "status": "ok"
    },
    "id": "0189dcd5-6dae-72c1-a04c-8caceba85a32",
    "lastModifiedBy": {
        "username": "api-key-pok_vipgj...bjjvyo",
        "userId": "a52cacf6-3ddc-48e5-8675-xxxxxxxxxxxx"
    },
    "lastUpdatedTimestamp": "2023-08-10T00:25:22.094458188Z",
    "spec": {
        "target": {
            "tableName": "Koalas",
            "type": "table",
            "intervals": []
        },
        "type": "drop_table",
        "desiredExecutionStatus": "running"
    },
    "target": {
        "tableName": "Koalas",
        "type": "table",
        "intervals": []
    },
    "type": "drop_table",
    "completedTimestamp": null,
    "startedTimestamp": null
}

Learn more

See the following topics for more information:

Recover data by API for recovering soft deleted data.
Delete data for deleting data using the UI.
Set a storage policy by API for automatically deleting data using a retention policy.

Delete behavior​

Prerequisites​

Soft delete data​

Soft delete data by time interval​

Sample request​

Sample response​

Soft delete all data​

Sample request​

Sample response​

Drop a table and soft delete its data​

Sample request​

Sample response​

Permanently delete data​

Permanently delete data by time interval​

Sample request​

Sample response​

Permanently delete all data​

Sample request​

Sample response​

Drop a table and permanently delete its data​

Sample request​

Sample response​

Learn more​

Delete behavior

Prerequisites

Soft delete data

Soft delete data by time interval

Sample request

Sample response

Soft delete all data

Sample request

Sample response

Drop a table and soft delete its data

Sample request

Sample response

Permanently delete data

Permanently delete data by time interval

Sample request

Sample response

Permanently delete all data

Sample request

Sample response

Drop a table and permanently delete its data

Sample request

Sample response

Learn more