Apache Druid
  • Imply Documentation

›Misc

Getting started

  • Introduction to Apache Druid
  • Quickstart
  • Docker
  • Single server deployment
  • Clustered deployment

Tutorials

  • Loading files natively
  • Load from Apache Kafka
  • Load from Apache Hadoop
  • Querying data
  • Roll-up
  • Configuring data retention
  • Updating existing data
  • Compacting segments
  • Deleting data
  • Writing an ingestion spec
  • Transforming input data
  • Kerberized HDFS deep storage

Design

  • Design
  • Segments
  • Processes and servers
  • Deep storage
  • Metadata storage
  • ZooKeeper

Ingestion

  • Ingestion
  • Data formats
  • Schema design tips
  • Data management
  • Stream ingestion

    • Apache Kafka
    • Amazon Kinesis
    • Tranquility

    Batch ingestion

    • Native batch
    • Hadoop-based
  • Task reference
  • Troubleshooting FAQ

Querying

  • Druid SQL
  • Native queries
  • Query execution
  • Concepts

    • Datasources
    • Joins
    • Lookups
    • Multi-value dimensions
    • Multitenancy
    • Query caching
    • Context parameters

    Native query types

    • Timeseries
    • TopN
    • GroupBy
    • Scan
    • Search
    • TimeBoundary
    • SegmentMetadata
    • DatasourceMetadata

    Native query components

    • Filters
    • Granularities
    • Dimensions
    • Aggregations
    • Post-aggregations
    • Expressions
    • Having filters (groupBy)
    • Sorting and limiting (groupBy)
    • Sorting (topN)
    • String comparators
    • Virtual columns
    • Spatial filters

Configuration

  • Configuration reference
  • Extensions
  • Logging

Operations

  • Web console
  • Getting started with Apache Druid
  • Basic cluster tuning
  • API reference
  • High availability
  • Rolling updates
  • Retaining or automatically dropping data
  • Metrics
  • Alerts
  • Working with different versions of Apache Hadoop
  • HTTP compression
  • TLS support
  • Password providers
  • dump-segment tool
  • reset-cluster tool
  • insert-segment-to-db tool
  • pull-deps tool
  • Misc

    • Legacy Management UIs
    • Deep storage migration
    • Export Metadata Tool
    • Metadata Migration
    • Segment Size Optimization
    • Content for build.sbt

Development

  • Developing on Druid
  • Creating extensions
  • JavaScript functionality
  • Build from source
  • Versioning
  • Experimental features

Misc

  • Papers

Hidden

  • Apache Druid vs Elasticsearch
  • Apache Druid vs. Key/Value Stores (HBase/Cassandra/OpenTSDB)
  • Apache Druid vs Kudu
  • Apache Druid vs Redshift
  • Apache Druid vs Spark
  • Apache Druid vs SQL-on-Hadoop
  • Authentication and Authorization
  • Broker
  • Coordinator Process
  • Historical Process
  • Indexer Process
  • Indexing Service
  • MiddleManager Process
  • Overlord Process
  • Router Process
  • Peons
  • Approximate Histogram aggregators
  • Apache Avro
  • Microsoft Azure
  • Bloom Filter
  • DataSketches extension
  • DataSketches HLL Sketch module
  • DataSketches Quantiles Sketch module
  • DataSketches Theta Sketch module
  • DataSketches Tuple Sketch module
  • Basic Security
  • Kerberos
  • Cached Lookup Module
  • Apache Ranger Security
  • Google Cloud Storage
  • HDFS
  • Apache Kafka Lookups
  • Globally Cached Lookups
  • MySQL Metadata Store
  • ORC Extension
  • Druid pac4j based Security extension
  • Apache Parquet Extension
  • PostgreSQL Metadata Store
  • Protobuf
  • S3-compatible
  • Simple SSLContext Provider Module
  • Stats aggregator
  • Test Stats Aggregators
  • Ambari Metrics Emitter
  • Apache Cassandra
  • Rackspace Cloud Files
  • DistinctCount Aggregator
  • Graphite Emitter
  • InfluxDB Line Protocol Parser
  • InfluxDB Emitter
  • Kafka Emitter
  • Materialized View
  • Moment Sketches for Approximate Quantiles module
  • Moving Average Query
  • OpenTSDB Emitter
  • Druid Redis Cache
  • Microsoft SQLServer
  • StatsD Emitter
  • T-Digest Quantiles Sketch module
  • Thrift
  • Timestamp Min/Max aggregators
  • GCE Extensions
  • Aliyun OSS
  • Cardinality/HyperUnique aggregators
  • Select
  • Realtime Process
Edit

Content for build.sbt

libraryDependencies ++= Seq(
  "com.amazonaws" % "aws-java-sdk" % "1.9.23" exclude("common-logging", "common-logging"),
  "org.joda" % "joda-convert" % "1.7",
  "joda-time" % "joda-time" % "2.7",
  "org.apache.druid" % "druid" % "0.8.1" excludeAll (
    ExclusionRule("org.ow2.asm"),
    ExclusionRule("com.fasterxml.jackson.core"),
    ExclusionRule("com.fasterxml.jackson.datatype"),
    ExclusionRule("com.fasterxml.jackson.dataformat"),
    ExclusionRule("com.fasterxml.jackson.jaxrs"),
    ExclusionRule("com.fasterxml.jackson.module")
  ),
  "org.apache.druid" % "druid-services" % "0.8.1" excludeAll (
    ExclusionRule("org.ow2.asm"),
    ExclusionRule("com.fasterxml.jackson.core"),
    ExclusionRule("com.fasterxml.jackson.datatype"),
    ExclusionRule("com.fasterxml.jackson.dataformat"),
    ExclusionRule("com.fasterxml.jackson.jaxrs"),
    ExclusionRule("com.fasterxml.jackson.module")
  ),
  "org.apache.druid" % "druid-indexing-service" % "0.8.1" excludeAll (
    ExclusionRule("org.ow2.asm"),
    ExclusionRule("com.fasterxml.jackson.core"),
    ExclusionRule("com.fasterxml.jackson.datatype"),
    ExclusionRule("com.fasterxml.jackson.dataformat"),
    ExclusionRule("com.fasterxml.jackson.jaxrs"),
    ExclusionRule("com.fasterxml.jackson.module")
  ),
  "org.apache.druid" % "druid-indexing-hadoop" % "0.8.1" excludeAll (
    ExclusionRule("org.ow2.asm"),
    ExclusionRule("com.fasterxml.jackson.core"),
    ExclusionRule("com.fasterxml.jackson.datatype"),
    ExclusionRule("com.fasterxml.jackson.dataformat"),
    ExclusionRule("com.fasterxml.jackson.jaxrs"),
    ExclusionRule("com.fasterxml.jackson.module")
  ),
  "org.apache.druid.extensions" % "mysql-metadata-storage" % "0.8.1" excludeAll (
    ExclusionRule("org.ow2.asm"),
    ExclusionRule("com.fasterxml.jackson.core"),
    ExclusionRule("com.fasterxml.jackson.datatype"),
    ExclusionRule("com.fasterxml.jackson.dataformat"),
    ExclusionRule("com.fasterxml.jackson.jaxrs"),
    ExclusionRule("com.fasterxml.jackson.module")
  ),
  "org.apache.druid.extensions" % "druid-s3-extensions" % "0.8.1" excludeAll (
    ExclusionRule("org.ow2.asm"),
    ExclusionRule("com.fasterxml.jackson.core"),
    ExclusionRule("com.fasterxml.jackson.datatype"),
    ExclusionRule("com.fasterxml.jackson.dataformat"),
    ExclusionRule("com.fasterxml.jackson.jaxrs"),
    ExclusionRule("com.fasterxml.jackson.module")
  ),
  "org.apache.druid.extensions" % "druid-histogram" % "0.8.1" excludeAll (
    ExclusionRule("org.ow2.asm"),
    ExclusionRule("com.fasterxml.jackson.core"),
    ExclusionRule("com.fasterxml.jackson.datatype"),
    ExclusionRule("com.fasterxml.jackson.dataformat"),
    ExclusionRule("com.fasterxml.jackson.jaxrs"),
    ExclusionRule("com.fasterxml.jackson.module")
  ),
  "org.apache.druid.extensions" % "druid-hdfs-storage" % "0.8.1" excludeAll (
    ExclusionRule("org.ow2.asm"),
    ExclusionRule("com.fasterxml.jackson.core"),
    ExclusionRule("com.fasterxml.jackson.datatype"),
    ExclusionRule("com.fasterxml.jackson.dataformat"),
    ExclusionRule("com.fasterxml.jackson.jaxrs"),
    ExclusionRule("com.fasterxml.jackson.module")
  ),
  "com.fasterxml.jackson.core" % "jackson-annotations" % "2.3.0",
  "com.fasterxml.jackson.core" % "jackson-core" % "2.3.0",
  "com.fasterxml.jackson.core" % "jackson-databind" % "2.3.0",
  "com.fasterxml.jackson.datatype" % "jackson-datatype-guava" % "2.3.0",
  "com.fasterxml.jackson.datatype" % "jackson-datatype-joda" % "2.3.0",
  "com.fasterxml.jackson.jaxrs" % "jackson-jaxrs-base" % "2.3.0",
  "com.fasterxml.jackson.jaxrs" % "jackson-jaxrs-json-provider" % "2.3.0",
  "com.fasterxml.jackson.jaxrs" % "jackson-jaxrs-smile-provider" % "2.3.0",
  "com.fasterxml.jackson.module" % "jackson-module-jaxb-annotations" % "2.3.0",
  "com.sun.jersey" % "jersey-servlet" % "1.17.1",
  "mysql" % "mysql-connector-java" % "5.1.34",
  "org.scalatest" %% "scalatest" % "2.2.3" % "test",
  "org.mockito" % "mockito-core" % "1.10.19" % "test"
)

assemblyMergeStrategy in assembly := {
  case path if path contains "pom." => MergeStrategy.first
  case path if path contains "javax.inject.Named" => MergeStrategy.first
  case path if path contains "mime.types" => MergeStrategy.first
  case path if path contains "org/apache/commons/logging/impl/SimpleLog.class" => MergeStrategy.first
  case path if path contains "org/apache/commons/logging/impl/SimpleLog$1.class" => MergeStrategy.first
  case path if path contains "org/apache/commons/logging/impl/NoOpLog.class" => MergeStrategy.first
  case path if path contains "org/apache/commons/logging/LogFactory.class" => MergeStrategy.first
  case path if path contains "org/apache/commons/logging/LogConfigurationException.class" => MergeStrategy.first
  case path if path contains "org/apache/commons/logging/Log.class" => MergeStrategy.first
  case path if path contains "META-INF/jersey-module-version" => MergeStrategy.first
  case path if path contains ".properties" => MergeStrategy.first
  case path if path contains ".class" => MergeStrategy.first
  case x =>
    val oldStrategy = (assemblyMergeStrategy in assembly).value
    oldStrategy(x)
}
← Segment Size OptimizationDeveloping on Druid →

Technology · Use Cases · Powered by Druid · Docs · Community · Download · FAQ

 ·  ·  · 
Copyright © 2019 Apache Software Foundation.
Except where otherwise noted, licensed under CC BY-SA 4.0.
Apache Druid, Druid, and the Druid logo are either registered trademarks or trademarks of The Apache Software Foundation in the United States and other countries.