There are two supported mechanisms for ingesting data from Kafka:
Druid Kafka indexing service, a newer Druid extension first available in Imply 1.3.0 / Druid 0.9.1.1. This service offers exactly-once ingestion guarantees as well as the ability to ingest historical data. Additionally, this runs as part of the core Druid services and does not require any additional processes. Setup instructions are available in our Kafka indexing service tutorial and the Druid Kafka indexing service documentation.
Both options support partitioning and replication, and can be easily scaled out to achieve the desired ingestion throughput. While we provide support for both ingestion methods, please note that we are still considering the Kafka indexing service to be experimental until it has been thoroughly battle-tested in larger production systems.