HDFS
To use this Apache Druid extension, include druid-hdfs-storage in the extensions load list and run druid processes with GOOGLE_APPLICATION_CREDENTIALS=/path/to/service_account_keyfile in the environment.
Deep Storage
Configuration for HDFS
| Property | Possible Values | Description | Default |
|---|---|---|---|
druid.storage.type | hdfs | Must be set. | |
druid.storage.storageDirectory | Directory for storing segments. | Must be set. | |
druid.hadoop.security.kerberos.principal | druid@EXAMPLE.COM | Principal user name | empty |
druid.hadoop.security.kerberos.keytab | /etc/security/keytabs/druid.headlessUser.keytab | Path to keytab file | empty |
Besides the above settings, you also need to include all Hadoop configuration files (such as core-site.xml, hdfs-site.xml)
in the Druid classpath. One way to do this is copying all those files under ${DRUID_HOME}/conf/_common.
If you want to eagerly authenticate against a secured hadoop/hdfs cluster you must set druid.hadoop.security.kerberos.principal and druid.hadoop.security.kerberos.keytab, this is an alternative to the cron job method that runs kinit command periodically.