Skip to main content

Restore data by API

info

Project-less regional API resources have been deprecated and will be removed by the end of September 2024.

You must include the project ID in the URL for all regional API calls in projects created after September 29, 2023. For example: https://ORGANIZATION_NAME.REGION.CLOUD_PROVIDER.api.imply.io/v1/projects/PROJECT_ID

Projects created before September 29, 2023 can continue to use project-less URLs until the end of September 2024. We strongly recommend updating your regional API calls to include the project ID prior to September 2024. See the API migration guide for more information.

You can use the Imply Polaris API to restore the following data deleted from a table as long as it is within the grace period:

info

Imply reserves the right to delete data before the end of 30 day grace period, if needed.

For information on recovering data using the UI, see Manage deleted data.

Prerequisites​

This topic assumes that you have the following:

  • A table that has data soft deleted within the past 30 days.
  • An API key with the ManageIngestionJobs and ManageTables permissions. In the examples below, the key value is stored in the variable named POLARIS_API_KEY. To obtain an API key and assign permissions, see API key authentication. For more information on permissions, visit Permissions reference.

Restore data​

Data restoration is a job in Polaris in which the job type is restore_data. The job request takes a root property, interval, and an ISO 8601 time interval of the data to recover.

While you can delete multiple intervals by providing an array of them, you can only restore a single interval at a time. For data that spans multiple time intervals, create a separate job for each one.

To view data that can be restored (soft deleted data), use the Tables API to send a GET request to the /unusedSegments endpoint. Polaris returns the soft deleted data as a list of segments that are not in use. For example, to view soft deleted data in a table called demo_table, make the following request:

GET /v1/projects/{projectId}/tables/demo_table/unusedSegments

Sample request​

The following example shows a restore_data job that recovers data for the time interval 2022-07-01/2022-08-01. See the Jobs v1 API documentation for more information.

curl --location --request POST "https://ORGANIZATION_NAME.REGION.CLOUD_PROVIDER.api.imply.io/v1/projects/PROJECT_ID/jobs" \
--user ${POLARIS_API_KEY}: \
--header 'Content-Type: application/json' \
--data '{
    "type": "restore_data",
    "target": {
        "type": "table",
        "tableName": "demo_table"
    },
    "interval": "2022-07-01/2022-08-01"
}'

Sample response​

The following example shows a successful response:

Click to view the response
{
    "deleteAll": false,
    "createdBy": {
        "username": "api-key-pok_vipgj...bjjvyo",
        "userId": "a52cacf6-3ddc-48e5-8675-xxxxxxxxxxxx"
    },
    "createdTimestamp": "2023-08-10T00:14:21.188501674Z",
    "desiredExecutionStatus": "running",
    "executionStatus": "pending",
    "health": {
        "status": "ok"
    },
    "id": "0189dccb-5804-7bf9-bbbb-6ac085b71b44",
    "lastModifiedBy": {
        "username": "api-key-pok_vipgj...bjjvyo",
        "userId": "a52cacf6-3ddc-48e5-8675-xxxxxxxxxxxx"
    },
    "lastUpdatedTimestamp": "2023-08-10T00:14:21.188501674Z",
    "spec": {
        "target": {
            "tableName": "demo_table",
            "type": "table",
            "intervals": [
                "2022-07-01/2022-08-01"
            ]
        },
        "deleteAll": false,
        "type": "restore_data",
        "desiredExecutionStatus": "running"
    },
    "target": {
        "tableName": "demo_table",
        "type": "table",
        "intervals": []
    },
    "type": "restore_data",
    "completedTimestamp": null,
    "startedTimestamp": null
}

Restore an older version of data that's in use​

If you want to restore a version of data that has a higher version in use, you need to first soft delete the newer overlapping data version that's in use.

Then, you can restore the earlier version as usual.

A common scenario that can require this process is when you want to restore segments that have been replaced. Polaris soft deletes older versions of segments when you replace data. Although you can restore the lower version, it gets soft deleted again automatically because there is a higher version in use.

You need to soft delete the higher version segment before you try to restore the lower version segment.

For example, if you have an interval, such as [2022/2023] and replace data twice in that interval, v2 of that data is in use and:

  • v0 was soft deleted and replaced with v1 as part of the first replace.
  • v1 was soft deleted and replaced with v2 as part of the second replace.

If you try to restore a v1 segment at this point, the v1 segment gets automatically soft deleted when it gets restored. This occurs because there is an existing v2 segment that corresponds to it in use already . To avoid that, soft delete the corresponding v2 segment first. Then, restore the v1 segment.

Learn more​

See the following topics for more information: