Imply Manager is provided as a pair of Docker images available on Docker Hub. There is one image for the manager container which includes the coordination services and manager application, and another image for the agent containers which become the nodes of one or more Druid clusters. At this time, a deployment will always have a single manager and multiple agents. Support for running multiple managers for high availability will be offered in the near future.
The images are available in the imply/onprem-manager and imply/onprem-agent Docker Hub repositories. These are private repositories, and you will need to provide your Docker ID to Imply in order to gain access.
The recommended way to configure the manager container is to create a configuration file that can be passed to the container using the --env-file Docker run argument. The remainder of this documentation will assume this file is named env.list.
The following items are required:
IMPLY_MANAGER_LICENSE_KEY={license_key}
IMPLY_MANAGER_STORE_TYPE=mysql
IMPLY_MANAGER_STORE_HOST={mysql_hostname}
IMPLY_MANAGER_STORE_PORT={mysql_port}
IMPLY_MANAGER_STORE_USER={mysql_user}
IMPLY_MANAGER_STORE_PASSWORD={mysql_password}
IMPLY_MANAGER_STORE_DATABASE=imply_managerNote: The database name (IMPLY_MANAGER_STORE_DATABASE) should not contain any spaces or dashes.
The Imply Manager application and the Druid clusters require an external MySQL database, version 5.7.21 or later. Ensure that this database will be reachable from the manager container and all agent containers.
You will also require a valid license key, which can be obtained by contacting a representative at Imply.
Druid clusters require:
You can provide default values for these external dependencies so that they don't need to be specified at cluster creation time. To do so, provide the following parameters in the env.list file:
imply_defaults_zkType=external
imply_defaults_zkHosts={zk-host-1}:2181,{zk-host-2}:2181,{zk-host-3}:2181
imply_defaults_zkBasePath=imply
imply_defaults_metadataStorageType=mysql
imply_defaults_metadataStorageHost={mysql_hostname}
imply_defaults_metadataStoragePort={mysql_port}
imply_defaults_metadataStorageUser={mysql_user}
imply_defaults_metadataStoragePassword={mysql_password}
imply_defaults_deepStorageType={'s3' / 'hdfs' / 'local'}
imply_defaults_deepStorageBaseLocation=s3://{bucket}/{prefix} # s3
# imply_defaults_deepStorageBaseLocation=hdfs://{namenode_host}:8020/{prefix} # hdfs
# imply_defaults_deepStorageBaseLocation=/mnt/var/druid/storage # local
# Not necessary if providing your credentials elsewhere, e.g. using an EC2 instance role
# imply_defaults_deepStorageS3AccessKey={s3_access_key}
# imply_defaults_deepStorageS3SecretKey={s3_secret_key}The agent containers only require a single configuration - setting the environment variable IMPLY_MANAGER_HOST to the hostname of the manager container. The agent will register with the manager and will pull any additional configurations that are required.
It is important that IMPLY_MANAGER_HOST be a name that is resolvable and routable from the agent containers. Appropriate values to use here depends on what Docker networking configuration you plan to use. For Linux-host deployments not utilizing a container orchestration system (Kubernetes, Docker Swarm, etc.), running a single container per host and starting the container with the --network host configuration is a common setup. In this case, IMPLY_MANAGER_HOST would be the externally addressable name or address of the host machine running the manager container. Note that --network host is not a supported option on OSX. See the Networking section for suggestions on how to run the container in OSX.
Your Docker run command will vary based on the networking configuration chosen. Note the following important ports:
For proper cluster coordination, the manager and agent containers should be able to talk to each other. In most default networking configurations, containers can talk to each other unrestricted. If you need to allow access on specific ports, the ones used for cluster coordination are:
The recommended deployment strategy if you are not using a container orchestration system and you are using a Linux host is to use host networking. In this configuration, the container will use the host's network stack, which will provide the best performance as there is no need for packets to be routed to a container's separate network stack. You would access the Imply Manager on port 9097 of the host system.
Note: Do not run multiple containers using host networking on a single host. You will encounter port conflicts and the agents will fail to properly register with the manager.
On a non-Linux host, you are unable to use host networking since the containers do not run natively on the host but run in a virtual machine. The recommended alternative is to use a user-defined bridge network. To create a bridge network, run:
docker network create implyYou will then be able to use this network when running containers by specifying --network imply and will be able to address a container from other containers by its container name.
Your Docker run command will also vary depending on if you plan to use Docker volumes or bind mounts. Note that by default, Docker runs containers on the host's primary storage device, which may not have sufficient space, particularly for Imply data nodes. You can either configure Docker to move these containers to a secondary device on the host, or use volumes/binds to mount another location inside the container. By default, Imply writes temporary files, logs, and caches to the /mnt directory, so this is a good path to mount a larger volume. See the Docker volume documentation for more information.
As an example, running the manager and agent using host networking, with a bind mount on /mnt, and using a configuration file named env.list would look like this. The commands must be run on different host machines:
docker run --rm --network host --env-file env.list --mount type=bind,source=/mnt,target=/mnt imply/onprem-managerdocker run --rm --network host -e "IMPLY_MANAGER_HOST=imply-manager" --mount type=bind,source=/mnt,target=/mnt imply/onprem-agentAs another example, running the manager and agent using a user-defined network named imply with a configuration file named env.list and the IMPLY_MANAGER_HOST property set to match a manager container name of 'imply-manager' would look like this:
docker run --rm --network imply --name imply-manager -p 9097:9097 --env-file env.list imply/onprem-managerdocker run --rm --network imply -p 9095 -p 8888 -e "IMPLY_MANAGER_HOST=imply-manager" imply/onprem-agentOnce the manager container is running, you can access the Imply Manager application at http://<managerHost>:9097
The manager can be configured to automatically generate certificates for the agents and itself by providing a signing certificate and key. This will allow the secure communication of the manager with the agents, the agents with each other, and users with the system. The key and certificate should be provided to the manager to generate the keys for itself and the agents. The agent will get the certificate only to validate the generated certificates.
Enabling, disabling or updating your certificates will require a restart of any running clusters. You will be notified in the manager that a certificate change has been detected and prompted to restart. You should plan the changes around maintenance windows as it will require an outage.
This ca.crt should be trusted/installed in browsers connecting to the manager or Druid so that they can be validated and the https:// scheme used for all
web requests.
When TLS is enabled Druid will use different ports by default as well as require the https:// schema for all web requests.
By default these ports are the default port for the service increased by 200. Note that Pivot and the manager still run on the same port.
See the table below for some examples:
| Service | Default Port | TLS Port |
|---|---|---|
| Imply Manager | 9097 | 9097 |
| Pivot | 9095 | 9095 |
| Router | 8888 | 9088 |
| Overlord | 8090 | 8290 |
Review the Druid Configuration reference for more defaults.
To generate a CA certificate and key you can use the following command:
openssl req -x509 -new -nodes -keyout ca.key -out ca.crt -days 365When executing the command you will be prompted to fill in the information for the certificate. For example:
Country Name (2 letter code) []:US
State or Province Name (full name) []:California
Locality Name (eg, city) []:Burlingame
Organization Name (eg, company) []:Imply Data, Inc
Organizational Unit Name (eg, section) []:
Common Name (eg, fully qualified host name) []:Imply Manager CA
Email Address []:These values are just an example and should be filled in with your own values. Once this is completed a new certificate
and key that are valid for 1 year will be generated. If you want a longer or shorter expiry update the -days argument. For more
information please consult the OpenSSL Documentation.
Double-clicking the ca.crt file will add it to your Keychain as a trusted certificate. For more information you can consult
the Keychain Access User Guide.
To add the certificate on windows we will perform the following steps.
ca.crt and select Install Certificate. This will launch the Certificate Import Wizard.For managed environments the certificate can be installed as part of a Group Policy.
By default the manager looks for the certificate files in the following locations:
The signing certificate:
/run/secrets/imply-ca.crt
The signing key:
/run/secrets/imply-ca.key
To enable TLS with Docker the certificates should be passed as volumes to Docker.
To run the manager providing the certificates as volumes the command would look like:
docker run --rm --network host --name imply-manager --env-file env.list -v path/to/ca.crt:/run/secrets/imply-ca.crt -v path/to/ca.key:/run/secrets/imply-ca.key imply/onprem-managerAnd to provide the certificate to the agent:
docker run --rm --network host -e "IMPLY_MANAGER_HOST=imply-manager" -v path/to/ca.crt:/run/secrets/imply-ca.crt imply/onprem-agentWhen the containers start you should see TLS: Enabled in the logs.
When using Docker Swarm we can use Docker secrets. To create the secret you can use the following command:
docker create secret imply-ca.crt path/to/ca.crt
docker create secret imply-ca.key path/to/ca.crtThen we can create the service and allow it access to the secret by us a command like:
docker service create --name imply-manager --env-file env.list --secret imply-ca.crt --secret imply-ca.key imply/onprem-manager
docker service create --name imply-agent -e "IMPLY_MANAGER_HOST=imply-manager" --secret imply-ca.crt imply/onprem-agentYou can verify that TLS was successfully enabled by running:
docker service logs imply-managerAnd verify that TLS: Enabled shows in the logs.
When using Kubernetes we can use Kubernetes secrets. To create the secret containing the certificate and key using the following command:
kubectl create secret tls imply-ca --key path/to/ca.key --cert path/to/ca.crtNow we will modify the imply/values.yaml to enable TLS. Uncomment the section:
security:
...
# tls:
# secretName: imply-caNow we can start or update the deployment. For more information on deploying to Helm review the Kubernetes Quickstart.
You can verify that TLS was successfully enabled by running:
kubectl logs -lapp.kubernetes.io/name=imply-manager --tail=10000And verify that TLS: Enabled shows in the logs.
Authentication can be enabled to allow the manager to communicate with the agents in an authenticated manner. It will also enable Druid and Pivot authentication to secure the entire deployment. This should be used in conjunction with TLS as the tokens are sent in plain text.
We will use the token supersecret in the following examples, but a different token should be generated and stored securely.
When authentication is enabled by providing a token to the manager and agents, it also enables Druid authentication by default. For more information on Druid authentication review https://druid.apache.org/docs/latest/design/auth.html.
When Druid Authentication is enabled you can find the credentials in the manager under the API tab.
When authentication is enabled by providing a token to the manager and agents, it also enables Pivot authentication by default. For more information on Pivot user modes review userMode configuration.
Pivot will be configured to use native-users. The default credentials can be found in the userMode configuration link above.
By default the manager looks for the authentication token in the following location:
/run/secrets/imply-auth-token
This file should contain one line, the shared secret to use. To create this file you could use the following command:
echo "supersecret" > path/to/auth-token
Note: Creating the authentication token in this way will leave the token in your history and is meant as an example only. Consider creating this file with an editor and ensure the permissions are as restricted as possible to keep it safe.
To enable authentication with Docker the authentication token should be passed as volumes to Docker.
To run the manager providing the authentication token as a volume the command would look like:
docker run --rm --network host --name imply-manager --env-file env.list -v path/to/auth-token:/run/secrets/imply-auth-token imply/onprem-managerAnd to provide the authentication token to the agent:
docker run --rm --network host -e "IMPLY_MANAGER_HOST=imply-manager" -v path/to/auth-token:/run/secrets/imply-auth-token imply/onprem-agentWhen the containers start you should see Authentication: Enabled in the logs.
When using Docker Swarm we can use Docker secrets. To create the secret you can use the following command:
echo "supersecret" | docker create secret imply-auth-token -Passing the authentication token in this way to agent will leave the token in your history and is meant as an example only. Please consult the Docker secrets documentation for secure ways to create the secret.
Then we can create the service and allow it access to the secret by us a command like:
docker service create --name imply-manager --env-file env.list --secret imply-auth-token imply/onprem-manager
docker service create --name imply-agent -e "IMPLY_MANAGER_HOST=imply-manager" --secret imply-auth-token imply/onprem-agentYou can verify that authentication was successfully enabled by running:
docker service logs imply-managerAnd verify that Authentication: Enabled shows in the logs.
When using Kubernetes we can use Kubernetes secrets. To create the secret containing the authentication token using the following command:
kubectl create secret generic imply-auth-token --from-literal="auth-token=supersecret"Passing the authentication token in this way to agent will leave the token in your history and is meant as an example only. Please consult the Kubernetes secrets documentation for secure ways to create the secret.
Now we will modify the imply/values.yaml to enable authentication. Uncomment the section:
security:
...
# auth:
# secretName: imply-auth-tokenNow we can start or update the deployment. For more information on deploying to Helm review the Kubernetes Quickstart.
You can verify that authentication was successfully enabled by running:
kubectl logs -lapp.kubernetes.io/name=imply-manager --tail=10000And verify that Authentication: Enabled shows in the logs.
Custom user files can be pushed by the manager to the nodes of the Imply cluster by specifying them in the Setup / Advanced Config section of the manager. These files can be written to the following locations:
/opt/imply/user)Additionally, these files can be processed as follows:
tar -x)chmod +x)There are two ways to make these files available to the manager:
manager:/// schemeTo make the files available to the manager locally, they will need to be placed in the /mnt/var/user directory inside the manager container. There are two ways to do this:
docker cp to copy the files from the host machine into the container/mnt/var/user. If using a bind mount during docker run, the argument would look similar to:--mount type=bind,source=/myfiles,target=/mnt/var/userOnce the files are available in that directory, reference them using the manager:/// scheme. As an example:
Imply Manager will automatically attempt to create the required databases if they do not exist. There is one database for the manager application and one database for each Imply cluster. In the case that the provided user does not have authorization to create these databases, you should create them manually prior to running the application. Both Druid and the Imply Manager expect a database with a 'utf8mb4' default character set. An example statement to create the manager database:
CREATE SCHEMA `imply_manager` DEFAULT CHARACTER SET utf8mb4;