Kubernetes Quickstart (preview)

Imply Manager is currently being offered as part of a preview program. To participate in this program, please contact your Imply representative.

This guide will walk through deploying the Imply Manager on a Kubernetes (K8s) cluster using an Imply-maintained Helm Chart. The commands assume that you are using OSX, but can be easily adapted for a Windows or Linux OS.

Prerequisites

To complete this quickstart, you will need:

• A Docker ID to access private images on Docker Hub (signup). You will need to provide this ID to Imply to be authorized to access the private repository.
• An Imply Manager license key. Contact your Imply representative for details on how to obtain this key.

Setup Environment

In this first section, we will setup a local single-node K8s cluster using Minikube that we can use for our Imply Manager deployment. You will need a cluster with a minimum of 2 CPUs and 6GB RAM available.

If you already have a K8s cluster available for use: skip to the section on installing Helm. If you already have Helm installed: skip to the section on adding the Imply repository.

Install Minikube

Install Minikube on your system according to the installation documentation. If you are using Homebrew, the main steps are:

• Install kubectl:

brew install kubernetes-cli

• Install a Hypervisor: This guide was written using VirtualBox.
• Install Minikube:

brew cask install minikube

Start Minikube

Start a local Kubernetes cluster with the required resources:

minikube start --cpus 2 --memory 6144 --disk-size 16g

Install Helm

Install Helm into your K8s cluster according to the installation documentation. If you are using Homebrew, the commands are:

brew install kubernetes-helm

The rest of this guide assumes that you are using Helm 3 or later.

Add Imply Repository

Add the Imply repository to Helm by running:

helm repo add imply https://static.imply.io/onprem/helm
helm repo update

Create Secret for Docker Registry

The Imply Manager images used by the Helm chart are stored in a private repository on Docker Hub. If you have not done so already, you will need to provide your Docker ID to Imply in order to gain access to this repository. Once this is done, you will need to generate a Secret to allow the nodes of your K8s cluster to pull this image. Generate a secret using the following command, substituting {dockerUsername}, {dockerEmail}, and {dockerPassword} with the appropriate values:

kubectl create secret docker-registry regcred --docker-server=https://index.docker.io/v1/ --docker-username={dockerUsername} --docker-email={dockerEmail} --docker-password={dockerPassword}

Create Secret for License Key

When running on K8s using Helm, there are two ways to provide your license key to Imply Manager. You may either supply it directly in Helm's values.yaml configuration file, or create it as a K8s Secret and provide Helm with a reference to the secret name. The latter method is more secure, and is the method we will be using here.

To create the secret containing the license key:

• Create a file named IMPLY_MANAGER_LICENSE_KEY and copy the license key as the contents of the file.
• Create a K8s secret named imply-secrets by running:

kubectl create secret generic imply-secrets --from-file=IMPLY_MANAGER_LICENSE_KEY

Install Imply Manager Chart

We are now ready to deploy our Imply chart. For this quickstart, we will install the default version of the chart which will create:

• a MySQL server (metadata storage for both the manager and the Imply cluster)
• a single-node ZooKeeper cluster
• a single-node MinIO server (as deep storage)
• Imply Manager
• an Imply cluster with:
  • 1 master node
  • 1 query node
  • 1 data node

In the Kubernetes Deployment Guide, we will build on this basic cluster by discussing how to configure and scale this chart to be a highly-available cluster suitable for a production deployment.

Install the chart by running:

helm install imply imply/imply

The chart will take a few moments to deploy, after which you will be presented with information on how to access your cluster:

NAME: imply
LAST DEPLOYED: Sun Dec 15 22:13:15 2019
NAMESPACE: default
STATUS: deployed
REVISION: 1
NOTES:
Release name: imply

It may take a few minutes for the deployment to become available.

Once all the pods are running, you can access the Manager console at http://127.0.0.1:9097 by running:
  kubectl port-forward svc/imply-manager-int 9097

Once the Imply Manager reports that the cluster is running, you can access Pivot at http://127.0.0.1:9095 by running:
  kubectl port-forward svc/imply-query 8888 9095

It may takes a few more minutes for the manager, dependencies, and initial Imply cluster to be deployed. Use the commands mentioned in the Helm Chart's notes to setup port forwarding to:

• Imply Manager (http://localhost:9097)
• Pivot (http://localhost:9095)
• Druid web console (http://localhost:8888).

If the manager or cluster is not available after a few minutes, use commands like kubectl get pods, kubectl describe pods, and kubectl logs -f -lapp.kubernetes.io/name=imply-manager --tail=1000 to check if the pods were created and are running properly.

Next Steps

Congratulations, you now have a managed Imply cluster! From here, you can:

• Continue with the Imply quickstart to load a sample data file.
• (More documentation coming soon)