Skip to main content

Druid SQL API

info

Apache Druid supports two query languages: Druid SQL and native queries. This document describes the SQL language.

In this topic, http://ROUTER_IP:ROUTER_PORT is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use http://localhost:8888 for quickstart deployments.

Query from Historicals

Submit a query

Submits a SQL-based query in the JSON request body. Returns a JSON object with the query results and optional metadata for the results. You can also use this endpoint to query metadata tables.

Each query has an associated SQL query ID. You can set this ID manually using the SQL context parameter sqlQueryId. If not set, Druid automatically generates sqlQueryId and returns it in the response header for X-Druid-SQL-Query-Id. Note that you need the sqlQueryId to cancel a query.

URL

POST /druid/v2/sql

Request body

The request body takes the following properties:

  • query: SQL query string.

  • resultFormat: String that indicates the format to return query results. Select one of the following formats:

    • object: Returns a JSON array of JSON objects with the HTTP response header Content-Type: application/json.
      Object field names match the columns returned by the SQL query in the same order as the SQL query.

    • array: Returns a JSON array of JSON arrays with the HTTP response header Content-Type: application/json.
      Each inner array has elements matching the columns returned by the SQL query, in order.

    • objectLines: Returns newline-delimited JSON objects with the HTTP response header Content-Type: text/plain.
      Newline separation facilitates parsing the entire response set as a stream if you don't have a streaming JSON parser. This format includes a single trailing newline character so you can detect a truncated response.

    • arrayLines: Returns newline-delimited JSON arrays with the HTTP response header Content-Type: text/plain.
      Newline separation facilitates parsing the entire response set as a stream if you don't have a streaming JSON parser. This format includes a single trailing newline character so you can detect a truncated response.

    • csv: Returns comma-separated values with one row per line. Sent with the HTTP response header Content-Type: text/csv.
      Druid uses double quotes to escape individual field values. For example, a value with a comma returns "A,B". If the field value contains a double quote character, Druid escapes it with a second double quote character. For example, foo"bar becomes foo""bar. This format includes a single trailing newline character so you can detect a truncated response.

  • header: Boolean value that determines whether to return information on column names. When set to true, Druid returns the column names as the first row of the results. To also get information on the column types, set typesHeader or sqlTypesHeader to true. For a comparative overview of data formats and configurations for the header, see the Query output format table.

  • typesHeader: Adds Druid runtime type information in the header. Requires header to be set to true. Complex types, like sketches, will be reported as COMPLEX<typeName> if a particular complex type name is known for that field, or as COMPLEX if the particular type name is unknown or mixed.

  • sqlTypesHeader: Adds SQL type information in the header. Requires header to be set to true.

    For compatibility, Druid returns the HTTP header X-Druid-SQL-Header-Included: yes when all of the following conditions are met:

    • The header property is set to true.
    • The version of Druid supports typesHeader and sqlTypesHeader, regardless of whether either property is set.
  • context: JSON object containing optional SQL query context parameters, such as to set the query ID, time zone, and whether to use an approximation algorithm for distinct count.

  • parameters: List of query parameters for parameterized queries. Each parameter in the array should be a JSON object containing the parameter's SQL data type and parameter value. For a list of supported SQL types, see Data types.

    For example:

    "parameters": [
    {
    "type": "VARCHAR",
    "value": "bar"
    }
    ]

Responses

Successfully submitted query

Client-side error handling and truncated responses

Druid reports errors that occur before the response body is sent as JSON with an HTTP 500 status code. The errors are reported using the same format as native Druid query errors. If an error occurs while Druid is sending the response body, the server handling the request stops the response midstream and logs an error.

This means that when you call the SQL API, you must properly handle response truncation. For object and array formats, truncated responses are invalid JSON. For line-oriented formats, Druid includes a newline character as the final character of every complete response. Absence of a final newline character indicates a truncated response.

If you detect a truncated response, treat it as an error.


Sample request

The following example retrieves all rows in the wikipedia datasource where the user is BlueMoon2662. The query is assigned the ID request01 using the sqlQueryId context parameter. The optional properties header, typesHeader, and sqlTypesHeader are set to true to include type information to the response.

curl "http://ROUTER_IP:ROUTER_PORT/druid/v2/sql" \
--header 'Content-Type: application/json' \
--data '{
"query": "SELECT * FROM wikipedia WHERE user='\''BlueMoon2662'\''",
"context" : {"sqlQueryId" : "request01"},
"header" : true,
"typesHeader" : true,
"sqlTypesHeader" : true
}'

Sample response

Click to show sample response
[
{
"__time": {
"type": "LONG",
"sqlType": "TIMESTAMP"
},
"channel": {
"type": "STRING",
"sqlType": "VARCHAR"
},
"cityName": {
"type": "STRING",
"sqlType": "VARCHAR"
},
"comment": {
"type": "STRING",
"sqlType": "VARCHAR"
},
"countryIsoCode": {
"type": "STRING",
"sqlType": "VARCHAR"
},
"countryName": {
"type": "STRING",
"sqlType": "VARCHAR"
},
"isAnonymous": {
"type": "LONG",
"sqlType": "BIGINT"
},
"isMinor": {
"type": "LONG",
"sqlType": "BIGINT"
},
"isNew": {
"type": "LONG",
"sqlType": "BIGINT"
},
"isRobot": {
"type": "LONG",
"sqlType": "BIGINT"
},
"isUnpatrolled": {
"type": "LONG",
"sqlType": "BIGINT"
},
"metroCode": {
"type": "LONG",
"sqlType": "BIGINT"
},
"namespace": {
"type": "STRING",
"sqlType": "VARCHAR"
},
"page": {
"type": "STRING",
"sqlType": "VARCHAR"
},
"regionIsoCode": {
"type": "STRING",
"sqlType": "VARCHAR"
},
"regionName": {
"type": "STRING",
"sqlType": "VARCHAR"
},
"user": {
"type": "STRING",
"sqlType": "VARCHAR"
},
"delta": {
"type": "LONG",
"sqlType": "BIGINT"
},
"added": {
"type": "LONG",
"sqlType": "BIGINT"
},
"deleted": {
"type": "LONG",
"sqlType": "BIGINT"
}
},
{
"__time": "2015-09-12T00:47:53.259Z",
"channel": "#ja.wikipedia",
"cityName": "",
"comment": "/* 対戦通算成績と得失点 */",
"countryIsoCode": "",
"countryName": "",
"isAnonymous": 0,
"isMinor": 1,
"isNew": 0,
"isRobot": 0,
"isUnpatrolled": 0,
"metroCode": 0,
"namespace": "Main",
"page": "アルビレックス新潟の年度別成績一覧",
"regionIsoCode": "",
"regionName": "",
"user": "BlueMoon2662",
"delta": 14,
"added": 14,
"deleted": 0
}
]

Cancel a query

Cancels a query on the Router or the Broker with the associated sqlQueryId. The sqlQueryId can be manually set when the query is submitted in the query context parameter, or if not set, Druid will generate one and return it in the response header when the query is successfully submitted. Note that Druid does not enforce a unique sqlQueryId in the query context. If you've set the same sqlQueryId for multiple queries, Druid cancels all requests with that query ID.

When you cancel a query, Druid handles the cancellation in a best-effort manner. Druid immediately marks the query as canceled and aborts the query execution as soon as possible. However, the query may continue running for a short time after you make the cancellation request.

Cancellation requests require READ permission on all resources used in the SQL query.

URL

DELETE /druid/v2/sql/:sqlQueryId

Responses

Successfully deleted query


Sample request

The following example cancels a request with the set query ID request01.

curl --request DELETE "http://ROUTER_IP:ROUTER_PORT/druid/v2/sql/request01"

Sample response

A successful response results in an HTTP 202 message code and an empty response body.

Query output format

The following table shows examples of how Druid returns the column names and data types based on the result format and the type request. In all cases, header is true. The examples includes the first row of results, where the value of user is BlueMoon2662.

| Format | typesHeader | sqlTypesHeader | Example output                                                                             |
|--------|-------------|----------------|--------------------------------------------------------------------------------------------|
| object | true | false | [ { "user" : { "type" : "STRING" } }, { "user" : "BlueMoon2662" } ] |
| object | true | true | [ { "user" : { "type" : "STRING", "sqlType" : "VARCHAR" } }, { "user" : "BlueMoon2662" } ] |
| object | false | true | [ { "user" : { "sqlType" : "VARCHAR" } }, { "user" : "BlueMoon2662" } ] |
| object | false | false | [ { "user" : null }, { "user" : "BlueMoon2662" } ] |
| array | true | false | [ [ "user" ], [ "STRING" ], [ "BlueMoon2662" ] ] |
| array | true | true | [ [ "user" ], [ "STRING" ], [ "VARCHAR" ], [ "BlueMoon2662" ] ] |
| array | false | true | [ [ "user" ], [ "VARCHAR" ], [ "BlueMoon2662" ] ] |
| array | false | false | [ [ "user" ], [ "BlueMoon2662" ] ] |
| csv | true | false | user STRING BlueMoon2662 |
| csv | true | true | user STRING VARCHAR BlueMoon2662 |
| csv | false | true | user VARCHAR BlueMoon2662 |
| csv | false | false | user BlueMoon2662 |

Query from deep storage

You can use the sql/statements endpoint to query segments that exist only in deep storage and are not loaded onto your Historical processes as determined by your load rules.

Note that at least one segment of a datasource must be available on a Historical process so that the Broker can plan your query. A quick way to check if this is true is whether or not a datasource is visible in the Druid console.

For more information, see Query from deep storage.

Submit a query

Submit a query for data stored in deep storage. Any data ingested into Druid is placed into deep storage. The query is contained in the "query" field in the JSON object within the request payload.

Note that at least part of a datasource must be available on a Historical process so that Druid can plan your query and only the user who submits a query can see the results.

URL

POST /druid/v2/sql/statements

Request body

Generally, the sql and sql/statements endpoints support the same response body fields with minor differences. For general information about the available fields, see Submit a query to the sql endpoint.

Keep the following in mind when submitting queries to the sql/statements endpoint:

  • Apart from the context parameters mentioned here there are additional context parameters for sql/statements specifically:

    • executionMode determines how query results are fetched. Druid currently only supports ASYNC. You must manually retrieve your results after the query completes.
    • selectDestination determines where final results get written. By default, results are written to task reports. Set this parameter to durableStorage to instruct Druid to write the results from SELECT queries to durable storage, which allows you to fetch larger result sets. For result sets with more than 3000 rows, it is highly recommended to use durableStorage. Note that this requires you to have durable storage for MSQ enabled.

Responses

Successfully queried from deep storage


Sample request

curl "http://ROUTER_IP:ROUTER_PORT/druid/v2/sql/statements" \
--header 'Content-Type: application/json' \
--data '{
"query": "SELECT * FROM wikipedia WHERE user='\''BlueMoon2662'\''",
"context": {
"executionMode":"ASYNC"
}
}'

Sample response

Click to show sample response
{
"queryId": "query-b82a7049-b94f-41f2-a230-7fef94768745",
"state": "ACCEPTED",
"createdAt": "2023-07-26T21:16:25.324Z",
"schema": [
{
"name": "__time",
"type": "TIMESTAMP",
"nativeType": "LONG"
},
{
"name": "channel",
"type": "VARCHAR",
"nativeType": "STRING"
},
{
"name": "cityName",
"type": "VARCHAR",
"nativeType": "STRING"
},
{
"name": "comment",
"type": "VARCHAR",
"nativeType": "STRING"
},
{
"name": "countryIsoCode",
"type": "VARCHAR",
"nativeType": "STRING"
},
{
"name": "countryName",
"type": "VARCHAR",
"nativeType": "STRING"
},
{
"name": "isAnonymous",
"type": "BIGINT",
"nativeType": "LONG"
},
{
"name": "isMinor",
"type": "BIGINT",
"nativeType": "LONG"
},
{
"name": "isNew",
"type": "BIGINT",
"nativeType": "LONG"
},
{
"name": "isRobot",
"type": "BIGINT",
"nativeType": "LONG"
},
{
"name": "isUnpatrolled",
"type": "BIGINT",
"nativeType": "LONG"
},
{
"name": "metroCode",
"type": "BIGINT",
"nativeType": "LONG"
},
{
"name": "namespace",
"type": "VARCHAR",
"nativeType": "STRING"
},
{
"name": "page",
"type": "VARCHAR",
"nativeType": "STRING"
},
{
"name": "regionIsoCode",
"type": "VARCHAR",
"nativeType": "STRING"
},
{
"name": "regionName",
"type": "VARCHAR",
"nativeType": "STRING"
},
{
"name": "user",
"type": "VARCHAR",
"nativeType": "STRING"
},
{
"name": "delta",
"type": "BIGINT",
"nativeType": "LONG"
},
{
"name": "added",
"type": "BIGINT",
"nativeType": "LONG"
},
{
"name": "deleted",
"type": "BIGINT",
"nativeType": "LONG"
}
],
"durationMs": -1
}

Get query status

Retrieves information about the query associated with the given query ID. The response matches the response from the POST API if the query is accepted or running and the execution mode is ASYNC. In addition to the fields that this endpoint shares with POST /sql/statements, a completed query's status includes the following:

  • A result object that summarizes information about your results, such as the total number of rows and sample records.
  • A pages object that includes the following information for each page of results:
    • numRows: the number of rows in that page of results.
    • sizeInBytes: the size of the page.
    • id: the page number that you can use to reference a specific page when you get query results.

URL

GET /druid/v2/sql/statements/:queryId

Responses

Successfully retrieved query status

Sample request

The following example retrieves the status of a query with specified ID query-9b93f6f7-ab0e-48f5-986a-3520f84f0804.

curl "http://ROUTER_IP:ROUTER_PORT/druid/v2/sql/statements/query-9b93f6f7-ab0e-48f5-986a-3520f84f0804"

Sample response

Click to show sample response
{
"queryId": "query-9b93f6f7-ab0e-48f5-986a-3520f84f0804",
"state": "SUCCESS",
"createdAt": "2023-07-26T22:57:46.620Z",
"schema": [
{
"name": "__time",
"type": "TIMESTAMP",
"nativeType": "LONG"
},
{
"name": "channel",
"type": "VARCHAR",
"nativeType": "STRING"
},
{
"name": "cityName",
"type": "VARCHAR",
"nativeType": "STRING"
},
{
"name": "comment",
"type": "VARCHAR",
"nativeType": "STRING"
},
{
"name": "countryIsoCode",
"type": "VARCHAR",
"nativeType": "STRING"
},
{
"name": "countryName",
"type": "VARCHAR",
"nativeType": "STRING"
},
{
"name": "isAnonymous",
"type": "BIGINT",
"nativeType": "LONG"
},
{
"name": "isMinor",
"type": "BIGINT",
"nativeType": "LONG"
},
{
"name": "isNew",
"type": "BIGINT",
"nativeType": "LONG"
},
{
"name": "isRobot",
"type": "BIGINT",
"nativeType": "LONG"
},
{
"name": "isUnpatrolled",
"type": "BIGINT",
"nativeType": "LONG"
},
{
"name": "metroCode",
"type": "BIGINT",
"nativeType": "LONG"
},
{
"name": "namespace",
"type": "VARCHAR",
"nativeType": "STRING"
},
{
"name": "page",
"type": "VARCHAR",
"nativeType": "STRING"
},
{
"name": "regionIsoCode",
"type": "VARCHAR",
"nativeType": "STRING"
},
{
"name": "regionName",
"type": "VARCHAR",
"nativeType": "STRING"
},
{
"name": "user",
"type": "VARCHAR",
"nativeType": "STRING"
},
{
"name": "delta",
"type": "BIGINT",
"nativeType": "LONG"
},
{
"name": "added",
"type": "BIGINT",
"nativeType": "LONG"
},
{
"name": "deleted",
"type": "BIGINT",
"nativeType": "LONG"
}
],
"durationMs": 25591,
"result": {
"numTotalRows": 1,
"totalSizeInBytes": 375,
"dataSource": "__query_select",
"sampleRecords": [
[
1442018873259,
"#ja.wikipedia",
"",
"/* 対戦通算成績と得失点 */",
"",
"",
0,
1,
0,
0,
0,
0,
"Main",
"アルビレックス新潟の年度別成績一覧",
"",
"",
"BlueMoon2662",
14,
14,
0
]
],
"pages": [
{
"id": 0,
"numRows": 1,
"sizeInBytes": 375
}
]
}
}

Get query results

Retrieves results for completed queries. Results are separated into pages, so you can use the optional page parameter to refine the results you get. Druid returns information about the composition of each page and its page number (id). For information about pages, see Get query status.

If a page number isn't passed, all results are returned sequentially in the same response. If you have large result sets, you may encounter timeouts based on the value configured for druid.router.http.readTimeout.

Getting the query results for an ingestion query returns an empty response.

URL

GET /druid/v2/sql/statements/:queryId/results

Query parameters

  • page (optional)
    • Type: Int
    • Fetch results based on page numbers. If not specified, all results are returned sequentially starting from page 0 to N in the same response.
  • resultFormat (optional)
    • Type: String
    • Defines the format in which the results are presented. The following options are supported arrayLines,objectLines,array,object, and csv. The default is object.

Responses

Successfully retrieved query results


Sample request

The following example retrieves the status of a query with specified ID query-f3bca219-173d-44d4-bdc7-5002e910352f.

curl "http://ROUTER_IP:ROUTER_PORT/druid/v2/sql/statements/query-f3bca219-173d-44d4-bdc7-5002e910352f/results"

Sample response

Click to show sample response
[
{
"__time": 1442018818771,
"channel": "#en.wikipedia",
"cityName": "",
"comment": "added project",
"countryIsoCode": "",
"countryName": "",
"isAnonymous": 0,
"isMinor": 0,
"isNew": 0,
"isRobot": 0,
"isUnpatrolled": 0,
"metroCode": 0,
"namespace": "Talk",
"page": "Talk:Oswald Tilghman",
"regionIsoCode": "",
"regionName": "",
"user": "GELongstreet",
"delta": 36,
"added": 36,
"deleted": 0
},
{
"__time": 1442018820496,
"channel": "#ca.wikipedia",
"cityName": "",
"comment": "Robot inserta {{Commonscat}} que enllaça amb [[commons:category:Rallicula]]",
"countryIsoCode": "",
"countryName": "",
"isAnonymous": 0,
"isMinor": 1,
"isNew": 0,
"isRobot": 1,
"isUnpatrolled": 0,
"metroCode": 0,
"namespace": "Main",
"page": "Rallicula",
"regionIsoCode": "",
"regionName": "",
"user": "PereBot",
"delta": 17,
"added": 17,
"deleted": 0
},
{
"__time": 1442018825474,
"channel": "#en.wikipedia",
"cityName": "Auburn",
"comment": "/* Status of peremptory norms under international law */ fixed spelling of 'Wimbledon'",
"countryIsoCode": "AU",
"countryName": "Australia",
"isAnonymous": 1,
"isMinor": 0,
"isNew": 0,
"isRobot": 0,
"isUnpatrolled": 0,
"metroCode": 0,
"namespace": "Main",
"page": "Peremptory norm",
"regionIsoCode": "NSW",
"regionName": "New South Wales",
"user": "60.225.66.142",
"delta": 0,
"added": 0,
"deleted": 0
},
{
"__time": 1442018828770,
"channel": "#vi.wikipedia",
"cityName": "",
"comment": "fix Lỗi CS1: ngày tháng",
"countryIsoCode": "",
"countryName": "",
"isAnonymous": 0,
"isMinor": 1,
"isNew": 0,
"isRobot": 1,
"isUnpatrolled": 0,
"metroCode": 0,
"namespace": "Main",
"page": "Apamea abruzzorum",
"regionIsoCode": "",
"regionName": "",
"user": "Cheers!-bot",
"delta": 18,
"added": 18,
"deleted": 0
},
{
"__time": 1442018831862,
"channel": "#vi.wikipedia",
"cityName": "",
"comment": "clean up using [[Project:AWB|AWB]]",
"countryIsoCode": "",
"countryName": "",
"isAnonymous": 0,
"isMinor": 0,
"isNew": 0,
"isRobot": 1,
"isUnpatrolled": 0,
"metroCode": 0,
"namespace": "Main",
"page": "Atractus flammigerus",
"regionIsoCode": "",
"regionName": "",
"user": "ThitxongkhoiAWB",
"delta": 18,
"added": 18,
"deleted": 0
},
{
"__time": 1442018833987,
"channel": "#vi.wikipedia",
"cityName": "",
"comment": "clean up using [[Project:AWB|AWB]]",
"countryIsoCode": "",
"countryName": "",
"isAnonymous": 0,
"isMinor": 0,
"isNew": 0,
"isRobot": 1,
"isUnpatrolled": 0,
"metroCode": 0,
"namespace": "Main",
"page": "Agama mossambica",
"regionIsoCode": "",
"regionName": "",
"user": "ThitxongkhoiAWB",
"delta": 18,
"added": 18,
"deleted": 0
},
{
"__time": 1442018837009,
"channel": "#ca.wikipedia",
"cityName": "",
"comment": "/* Imperi Austrohongarès */",
"countryIsoCode": "",
"countryName": "",
"isAnonymous": 0,
"isMinor": 0,
"isNew": 0,
"isRobot": 0,
"isUnpatrolled": 0,
"metroCode": 0,
"namespace": "Main",
"page": "Campanya dels Balcans (1914-1918)",
"regionIsoCode": "",
"regionName": "",
"user": "Jaumellecha",
"delta": -20,
"added": 0,
"deleted": 20
},
{
"__time": 1442018839591,
"channel": "#en.wikipedia",
"cityName": "",
"comment": "adding comment on notability and possible COI",
"countryIsoCode": "",
"countryName": "",
"isAnonymous": 0,
"isMinor": 0,
"isNew": 1,
"isRobot": 0,
"isUnpatrolled": 1,
"metroCode": 0,
"namespace": "Talk",
"page": "Talk:Dani Ploeger",
"regionIsoCode": "",
"regionName": "",
"user": "New Media Theorist",
"delta": 345,
"added": 345,
"deleted": 0
},
{
"__time": 1442018841578,
"channel": "#en.wikipedia",
"cityName": "",
"comment": "Copying assessment table to wiki",
"countryIsoCode": "",
"countryName": "",
"isAnonymous": 0,
"isMinor": 0,
"isNew": 0,
"isRobot": 1,
"isUnpatrolled": 0,
"metroCode": 0,
"namespace": "User",
"page": "User:WP 1.0 bot/Tables/Project/Pubs",
"regionIsoCode": "",
"regionName": "",
"user": "WP 1.0 bot",
"delta": 121,
"added": 121,
"deleted": 0
},
{
"__time": 1442018845821,
"channel": "#vi.wikipedia",
"cityName": "",
"comment": "clean up using [[Project:AWB|AWB]]",
"countryIsoCode": "",
"countryName": "",
"isAnonymous": 0,
"isMinor": 0,
"isNew": 0,
"isRobot": 1,
"isUnpatrolled": 0,
"metroCode": 0,
"namespace": "Main",
"page": "Agama persimilis",
"regionIsoCode": "",
"regionName": "",
"user": "ThitxongkhoiAWB",
"delta": 18,
"added": 18,
"deleted": 0
}
]

Cancel a query

Cancels a running or accepted query.

URL

DELETE /druid/v2/sql/statements/:queryId

Responses

A no op operation since the query is not in a state to be cancelled


Sample request

The following example cancels a query with specified ID query-945c9633-2fa2-49ab-80ae-8221c38c024da.

curl --request DELETE "http://ROUTER_IP:ROUTER_PORT/druid/v2/sql/statements/query-945c9633-2fa2-49ab-80ae-8221c38c024da"

Sample response

A successful request returns an HTTP 202 ACCEPTED message code and an empty response body.