Using the Imply Manager

The Imply Manager is a browser-based interface for deploying and administering an Imply cluster. It is available in Imply Cloud and licensed Imply Private deployments.

Among other tasks, you can use it to create and configure clusters and servers, monitor your deployment, and apply version updates to the cluster.

The following sections provide more details on using the Imply Manager.

Creating a cluster

An Imply cluster is made up of the servers that constitute a complete Imply deployment, including data servers, query servers, and master servers.

The way you create and scale a cluster differs depending on how you are running Imply. In an Imply Cloud installation, you can add clusters from the Imply Manager home page, as described in the Quickstart.

With Kubernetes, you can define the cluster to be created using a Helm chart. Servers are automatically added to the cluster when the Kubernetes scheduler creates new pods. For more information on cluster management with Kubernetes, see Deploy with Kubernetes.

Viewing cluster status and capacity usage

The cluster overview page lets you view the status and health of the cluster at a glance. The top of the page shows the current status of the cluster, capacity usage, and the servers that make up the cluster.

The capacity reflects the disk usage of the cluster, or more specifically, the segment cache usage compared to the segment cache size for the cluster as a whole. This information helps you determine when you need to scale the cluster or take other remediation measures.

imply manager cluster status

In considering disk usage, note that a certain amount of disk space is reserved for internal Imply operations, and therefore shown as unavailable in the disk usage graph. Specifically, about 25 GB is reserved for purposes of temporary storage space for ingestion, heap dumps, log files, and for other internal processes.

Administering a cluster

You can stop or start servers in the cluster, view process logs, and more from the Manager UI:

imply manager new servers

Updating software versions

The Imply Manager makes it easier to update software across a cluster.

If the cluster is running in highly availability mode, you can apply a rolling update. With a rolling update, servers are updated one at a time, ensuring that servers remain running to maintain service availability while undergoing the update.

Rolling updates are not supported for clusters with more than one Imply service (master, query, data) on a single node.

If your cluster is not highly available, updating versions will interrupt service for your users for a few minutes.

To update software:

  1. Open the cluster overview page and then Setup. Notice that the current version is listed on the Setup page, with the available versions appearing in the Imply versions menu.
  2. In the setup page, choose the version you want to update to from the Imply version menu. The Imply Manager gets and displays the version number of the latest releases automatically.
  3. Click Apply changes.
  4. Confirm the settings and apply the update.

The update status appears in the cluster overview page. At any point you can cancel the update, which causes the Manager to revert the upgrade, including the servers for which the update is complete.

Adding users and roles

You can add users and manage permissions for users through roles. Imply Manager includes built-in roles, and you can define your own.

You can access user settings from the Master settings configuration, accessible from the top left menu.

User permissions

Roles are composed of permissions. Note that users can only assign or manage other permissions that the users themselves have been assigned. That is, a user with the User Manager role only cannot grant other users the Super admin role.

Permissions in Imply Manager are:

  • ManageUsers – Enables users to create, remove, and modify user accounts in the Imply Manager UI, as well as assign users roles.
  • ManageUserRoles – Allows users to create, delete, and modify user roles.
  • ManagePasswords – Allows users to reset passwords for other users. When you trigger a password reset for another user, the Imply Manager provides a URL where the end user can enter a new password.
  • ManageUserStates – Allows users to unlock and enable access for other users.
  • ManageClusters – Allows users to create, modify, and remove clusters, including the ability to add servers to the cluster.

User roles

The Manager UI includes these predefined roles:

  • Super admin – Has all permissions except ManageUserStates.
  • User Manager – Has the following permissions:
    • ManageUsers
    • ManageUserRoles
    • ManagePasswords
    • ManageUserStates
  • Cluster Manager – Has ManageClusters permissions.

Configure extensions

You can see which Druid extensions are already included for your cluster, along with those that are available to be enabled, in the Druid extensions dialog. To do so, from the Advanced config section of the cluster setup page, click the edit icon next to the Druid extensions field.

At the top of the dialog box are names of the Druid extensions already included. Click a name for more information on that extension.

See Imply Cluster Extensions and Extensions in the Apache Druid documentation for more information on extensions.

Add custom user files

You can push custom user files to Imply cluster nodes by specifying them in the Setup / Advanced Config section of the manager. These files can be written to the following locations:

  • User files (/opt/imply/user)
  • Druid's classpath
  • as a Hadoop dependency

Additionally, these files can be processed as follows:

  • None
  • Requires unpacking (extracts using tar -x)
  • Is executable (sets chmod +x)

There are two ways to make these files available to the manager:

  • Make them available via HTTP(S) and provide the URL
  • Make them available to the manager locally and reference them using the manager:/// scheme

To make the files available to the manager locally, they will need to be placed in the /mnt/var/user directory inside the manager container.

Once the files are available in that directory, reference them using the manager:/// scheme. As an example:

  • File: /mnt/var/user/hdfs-site.xml -> Manager path: manager:///hdfs-site.xml

Lookups

The Lookups page lets you manage Druid lookups. For more information on lookups, see the Apache Druid documentation.

Experimental features

Experimental features are used to try out early access or other limited access features. Do not use except in consultation with Imply support.

Default server properties

The default values are intended for running a quickstart cluster on a moderately powerful desktop or laptop; they are not intended for production or large scale testing systems.

This section lists the default service settings for the cluster if you created using Docker for Imply Manager. They are provided here for reference purposes or if you need to set your cluster to its original state. (Note that if you generated your cluster through a Helm chart with Kubernetes, the default values are specified in the Helm file.)

When you apply any server property changes, an update (rolling or otherwise) is triggered; you'll have to proceed with the update to apply the changes.

  • Coordinator

    jvm.config.xms=-Xms128m
jvm.config.xmx=-Xmx128m

  • Overlord

    jvm.config.xms=-Xms128m
jvm.config.xmx=-Xmx128m

  • Broker

    jvm.config.xms=-Xms256m
jvm.config.xmx=-Xmx256m
druid.processing.buffer.sizeBytes=50000000
druid.processing.numThreads=1

  • Router

    jvm.config.xms=-Xms64m
jvm.config.xmx=-Xmx64m

  • Historical

    jvm.config.xms=-Xms256m
jvm.config.xmx=-Xmx256m
druid.processing.buffer.sizeBytes=50000000
druid.processing.numThreads=1
druid.segmentCache.locations=[{"path":"/mnt/var/druid/segment-cache","maxSize":10000000000}]
druid.server.maxSize=10000000000
druid.historical.cache.useCache=true
druid.historical.cache.populateCache=true
druid.cache.sizeInBytes=50000000

  • MiddleManager

    jvm.config.xms=-Xms64m
jvm.config.xmx=-Xmx64m
druid.worker.capacity=1
druid.indexer.runner.javaOpts=-server -Xmx2g -Duser.timezone=UTC -Dfile.encoding=UTF-8 -XX:MaxDirectMemorySize=3g -XX:+PrintGC -XX:+PrintGCDateStamps -XX:+ExitOnOutOfMemoryError -XX:+HeapDumpOnOutOfMemoryError -Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager -Daws.region=us-east-1
druid.processing.buffer.sizeBytes=50000000
druid.processing.numThreads=1
Imply Manager security →