Skip to main content

Asynchronous SQL download API

Asynchronous download is deprecated.

Apache Druid supports two query languages: Druid SQL and native queries. This document describes the SQL language.

Asynchronous query download (async download) lets you run longer-executing queries and retrieve the results after the queries complete. It solves problems caused when timeouts cause interruptions in the connection between query clients and the Druid cluster.

Query execution using the async download APIs is similar to that of using the synchronous APIs. Instead of one synchronous API, async download requires you to call three APIs to:

  • Submit the query.
  • Poll for query status.
  • Fetch the result.

API endpoints​

Use the async download API to submit and manage your asynchronous queries. As a prerequisite, enable async download as described in Async SQL download setup.

Submit a query​

Request​

To start a query, send a POST request with a request payload in the same format as the SQL sync API.

POST /druid/v2/sql/async/

Success response​

When the query has been accepted, Druid returns an HTTP 202 response and a JSON object with the following fields:

  • asyncResultId: Unique ID for the query. This ID may be different from the queryId for native or sql queries. Always present.
  • state: Query state. Always present. Possible states:
    • INITIALIZED: The query is set up but has not yet started running.
    • RUNNING: The query is running. The load state of the system affects the time it takes for a query to move from INITIALIZED to RUNNING.
    • COMPLETE: The query is finished, and results are ready to fetch.
    • FAILED: The query failed.
    • UNDETERMINED: The query state is unknown. Druid accepted the query but cannot determine its state. This may occur if the Broker has gone offline longer than the value specified in druid.query.async.cleanup.timeToWaitAfterBrokerGone.
  • resultFormat: Result format for this query. See the SQL sync API for the available result formats. Present when state is COMPLETE, absent otherwise.
  • resultLength: Size in bytes of the query results. Present when state is COMPLETE, absent otherwise.
  • error: Druid query error object, as described in Query execution failures. Present when state is FAILED, absent otherwise.

Error responses​

  • HTTP 429. Druid rejected the query due to concurrency limits. Retry the query after waiting for some time, for example 60 seconds. We recommend exponential backoff. Druid sets the query state to FAILED.

  • HTTP 4xx or 5xx. Druid rejected the query for a reason other than concurrency. HTTP 429 will not be returned in this case. Druid sets the query state to FAILED.

Get query status​

Send a GET request with the query ID as shown below to check the status of the query.

Request​

GET /druid/v2/sql/async/{asyncResultId}/status

Success response​

When the API succeeds, it returns an HTTP 200 response with the same JSON object as described in Submit a query.

Error responses​

Get query results​

Once a query completes, fetch the results by sending a GET request with the query ID.

Druid automatically cleans up expired query states and result files. For large results with long downloads, Druid periodically resets the retention period during the download to prevent premature cleanup of results. See Query state and result file management to configure the retention and refresh times for async downloads.

Request​

GET /druid/v2/sql/async/{asyncResultId}/results

Success response​

For a successful request, Druid returns an HTTP 200 response with the result file. The response contains the following HTTP headers:

  • Content-Length: the result file size
  • Content-Type: the query result format
  • Content-Disposition: "attachment"

This API returns the same results for the same query during the async download retention period, no matter how many times you call the API.

Error responses​

  • HTTP 403. The user is not authorized to get the query results. Only the user who submitted the query may view its results.

  • HTTP 404. Druid returns an HTTP 404 response for one of the following conditions:

    1. The async result ID does not exist. This may result if Druid rejected the query or if the async result ID expired. To configure the query retention time, see Query state and result file management.
    2. The query state is not COMPLETE. Check the state of the query using the query status API.

Cancel a query​

After you submit a query, you can subsequently cancel it using a DELETE request. Canceling a query cleans up all associated records as if you never issued the query. You can cancel queries of any state.

Request​

DELETE /druid/v2/sql/async/{asyncResultId}

Success response​

Druid returns an HTTP 202 response when it accepts the deletion request.

The cancellation proceeds in the background. Immediately after cancellation, the query status API may still return details for the query.

Error responses​

  • HTTP 403. The query ID originated from a different user. No one, including cluster admins, may cancel queries submitted from a different user.

  • HTTP 404. Druid returns an HTTP 404 response if the async result ID does not exist. This may result if Druid rejected the query or if the async result ID expired. To configure the query retention time, see Query state and result file management.

  • HTTP 500. Internal server error. Example:

    {
      "asyncResultId": "xxxx",
      "error": "unable to clear metadata"
    }

Learn more​