Skip to main content

Dynamic configuration API

This document describes the API endpoints to retrieve and manage dynamic configurations for the Coordinator and Overlord in Apache Druid.

In this topic, http://ROUTER_IP:ROUTER_PORT is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use http://localhost:8888 for quickstart deployments.

Coordinator dynamic configuration​

The Coordinator has dynamic configurations to tune certain behavior on the fly, without requiring a service restart. For information on the supported properties, see Coordinator dynamic configuration.

Get dynamic configuration​

Retrieves the current Coordinator dynamic configuration. Returns a JSON object with the dynamic configuration properties.

URL​

GET /druid/coordinator/v1/config

Responses​

Successfully retrieved dynamic configuration

Sample request​

curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config"

Sample response​

Click to show sample response
{
    "millisToWaitBeforeDeleting": 900000,
    "mergeBytesLimit": 524288000,
    "mergeSegmentsLimit": 100,
    "maxSegmentsToMove": 100,
    "replicantLifetime": 15,
    "replicationThrottleLimit": 500,
    "balancerComputeThreads": 1,
    "killDataSourceWhitelist": [],
    "killPendingSegmentsSkipList": [],
    "maxSegmentsInNodeLoadingQueue": 500,
    "decommissioningNodes": [],
    "decommissioningMaxPercentOfMaxSegmentsToMove": 70,
    "pauseCoordination": false,
    "replicateAfterLoadTimeout": false,
    "maxNonPrimaryReplicantsToLoad": 2147483647,
    "useRoundRobinSegmentAssignment": true,
    "smartSegmentLoading": true,
    "debugDimensions": null
}

Update dynamic configuration​

Submits a JSON-based dynamic configuration spec to the Coordinator. For information on the supported properties, see Dynamic configuration.

URL​

POST /druid/coordinator/v1/config

Header parameters​

The endpoint supports a set of optional header parameters to populate the author and comment fields in the configuration history.

  • X-Druid-Author
    • Type: String
    • Author of the configuration change.
  • X-Druid-Comment
    • Type: String
    • Description for the update.

Responses​

Successfully updated dynamic configuration

Sample request​

curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config" \
--header 'Content-Type: application/json' \
--data '{
  "millisToWaitBeforeDeleting": 900000,
  "mergeBytesLimit": 524288000,
  "mergeSegmentsLimit": 100,
  "maxSegmentsToMove": 5,
  "percentOfSegmentsToConsiderPerMove": 100,
  "useBatchedSegmentSampler": true,
  "replicantLifetime": 15,
  "replicationThrottleLimit": 10,
  "balancerComputeThreads": 1,
  "emitBalancingStats": true,
  "killDataSourceWhitelist": [],
  "killPendingSegmentsSkipList": [],
  "maxSegmentsInNodeLoadingQueue": 100,
  "decommissioningNodes": [],
  "decommissioningMaxPercentOfMaxSegmentsToMove": 70,
  "pauseCoordination": false,
  "replicateAfterLoadTimeout": false,
  "maxNonPrimaryReplicantsToLoad": 2147483647,
  "useRoundRobinSegmentAssignment": true
}'

Sample response​

A successful request returns an HTTP 200 OK message code and an empty response body.

Get dynamic configuration history​

Retrieves the history of changes to Coordinator dynamic configuration over an interval of time. Returns an empty array if there are no history records available.

URL​

GET /druid/coordinator/v1/config/history

Query parameters​

The endpoint supports a set of optional query parameters to filter results.

  • interval

    • Type: String
    • Limit the results to the specified time interval in ISO 8601 format delimited with /. For example, 2023-07-13/2023-07-19. The default interval is one week. You can change this period by setting druid.audit.manager.auditHistoryMillis in the runtime.properties file for the Coordinator.

  • count

    • Type: Integer
    • Limit the number of results to the last n entries.

Responses​

Successfully retrieved history

Sample request​

The following example retrieves the dynamic configuration history between 2022-07-13 and 2024-07-19. The response is limited to 10 entries.

curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/history?interval=2022-07-13%2F2024-07-19&count=10"

Sample response​

Click to show sample response
[
    {
        "key": "coordinator.config",
        "type": "coordinator.config",
        "auditInfo": {
            "author": "",
            "comment": "",
            "ip": "127.0.0.1"
        },
        "payload": "{\"millisToWaitBeforeDeleting\":900000,\"mergeBytesLimit\":524288000,\"mergeSegmentsLimit\":100,\"maxSegmentsToMove\":5,\"replicantLifetime\":15,\"replicationThrottleLimit\":10,\"balancerComputeThreads\":1,\"killDataSourceWhitelist\":[],\"killPendingSegmentsSkipList\":[],\"maxSegmentsInNodeLoadingQueue\":100,\"decommissioningNodes\":[],\"decommissioningMaxPercentOfMaxSegmentsToMove\":70,\"pauseCoordination\":false,\"replicateAfterLoadTimeout\":false,\"maxNonPrimaryReplicantsToLoad\":2147483647,\"useRoundRobinSegmentAssignment\":true,\"smartSegmentLoading\":true,\"debugDimensions\":null}",
        "auditTime": "2023-10-03T20:59:51.622Z"
    }
]

Overlord dynamic configuration​

The Overlord has dynamic configurations to tune how Druid assigns tasks to workers. For information on the supported properties, see Overlord dynamic configuration.

Get dynamic configuration​

Retrieves the current Overlord dynamic configuration. Returns a JSON object with the dynamic configuration properties. Returns an empty response body if there is no current Overlord dynamic configuration.

URL​

GET /druid/indexer/v1/worker

Responses​

Successfully retrieved dynamic configuration

Sample request​

curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/worker"

Sample response​

Click to show sample response
{
    "type": "default",
    "selectStrategy": {
        "type": "fillCapacityWithCategorySpec",
        "workerCategorySpec": {
            "categoryMap": {},
            "strong": true
        }
    },
    "autoScaler": null
}

Update dynamic configuration​

Submits a JSON-based dynamic configuration spec to the Overlord. For information on the supported properties, see Overlord dynamic configuration.

URL​

POST/druid/indexer/v1/worker

Header parameters​

The endpoint supports a set of optional header parameters to populate the author and comment fields in the configuration history.

  • X-Druid-Author
    • Type: String
    • Author of the configuration change.
  • X-Druid-Comment
    • Type: String
    • Description for the update.

Responses​

Successfully updated dynamic configuration

Sample request​

curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/worker" \
--header 'Content-Type: application/json' \
--data '{
  "type": "default",
  "selectStrategy": {
    "type": "fillCapacityWithCategorySpec",
    "workerCategorySpec": {
      "categoryMap": {},
      "strong": true
    }
  },
  "autoScaler": null
}'

Sample response​

A successful request returns an HTTP 200 OK message code and an empty response body.

Get dynamic configuration history​

Retrieves the history of changes to Overlord dynamic configuration over an interval of time. Returns an empty array if there are no history records available.

URL​

GET /druid/indexer/v1/worker/history

Query parameters​

The endpoint supports a set of optional query parameters to filter results.

  • interval

    • Type: String
    • Limit the results to the specified time interval in ISO 8601 format delimited with /. For example, 2023-07-13/2023-07-19. The default interval is one week. You can change this period by setting druid.audit.manager.auditHistoryMillis in the runtime.properties file for the Overlord.

  • count

    • Type: Integer
    • Limit the number of results to the last n entries.

Responses​

Successfully retrieved history

Sample request​

The following example retrieves the dynamic configuration history between 2022-07-13 and 2024-07-19. The response is limited to 10 entries.

curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/worker/history?interval=2022-07-13%2F2024-07-19&count=10"

Sample response​

Click to show sample response
[
    {
        "key": "worker.config",
        "type": "worker.config",
        "auditInfo": {
            "author": "",
            "comment": "",
            "ip": "127.0.0.1"
        },
        "payload": "{\"type\":\"default\",\"selectStrategy\":{\"type\":\"fillCapacityWithCategorySpec\",\"workerCategorySpec\":{\"categoryMap\":{},\"strong\":true}},\"autoScaler\":null}",
        "auditTime": "2023-10-03T21:49:49.991Z"
    }
]

Get an array of worker nodes in the cluster​

Returns an array of all the worker nodes in the cluster along with its corresponding metadata.

GET/druid/indexer/v1/workers

Responses​

Successfully retrieved worker nodes

Sample request​

curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/workers"

Sample response​

Click to show sample response
[
    {
        "worker": {
            "scheme": "http",
            "host": "localhost:8091",
            "ip": "198.51.100.0",
            "capacity": 2,
            "version": "0",
            "category": "_default_worker_category"
        },
        "currCapacityUsed": 0,
        "currParallelIndexCapacityUsed": 0,
        "availabilityGroups": [],
        "runningTasks": [],
        "lastCompletedTaskTime": "2023-09-29T19:13:05.505Z",
        "blacklistedUntil": null
    }
]

Get scaling events​

Returns Overlord scaling events if autoscaling runners are in use. Returns an empty response body if there are no Overlord scaling events.

URL​

GET/druid/indexer/v1/scaling

Responses​

Successfully retrieved scaling events

Sample request​

curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/scaling"

Sample response​

A successful request returns a 200 OK response and an array of scaling events.