Skip to main content

Configure node labels and taints for Lumi Enterprise on OpenShift

info

This information applies to Imply Lumi Enterprise deployments on OpenShift.
Lumi Enterprise on OpenShift is an early access feature.

Imply Lumi Enterprise groups internal services into node types, such as data, broker, and ingestion. You can use Kubernetes node labels, taints, and scheduling settings to control which OpenShift nodes run the pods for each node type.

This topic explains how to apply labels and taints to OpenShift nodes and how to configure Lumi pods to use them. For more information on each node type, see Size and prepare nodes.

How labels and taints work​

Kubernetes uses the following mechanisms to influence where a pod runs:

  • Node label: A key-value pair you attach to a node. You use labels to attach pods to specific nodes or groups of nodes, so that those pods only run on nodes with certain properties. For more information, see the Kubernetes documentation on assigning pods to nodes.
  • Taint: A marker you apply to a node to exclude pods without the required toleration value. This way taints prevent certain pods from loading on a node. For more information, see the Kubernetes documentation on taints and tolerations.

Lumi doesn't set default node selectors, affinities, tolerations, or topology spread constraints on its pods, so it doesn't require any particular labels or taints on your nodes. If you taint an OpenShift node, Lumi pods can't run on it unless you also configure matching tolerations.

Control where Lumi pods run​

To run a node type on specific OpenShift nodes, label and optionally taint those nodes, then configure the node type in your Helm values. Each node type has its own key in the Helm values: data, virtual, coordinator, broker, and ingest. For more information about each node type, see Size and prepare nodes.

For each node type, you can set the following values:

  • nodeSelector: Runs the pods only on nodes with matching labels.
  • affinity: Sets more flexible placement rules than nodeSelector, such as preferred rather than required nodes.
  • tolerations: Lets the pods run on nodes with matching taints.
  • topologySpreadConstraints: Spreads the pods across nodes or failure domains, such as racks or zones.

For more information, see the Kubernetes documentation on pod topology spread constraints.

The following example reserves a node for coordinator pods.

First, label and taint the node:

export NODE_NAME=worker-coordinator-1
oc label node ${NODE_NAME} example.com/lumi-role=coordinator
oc adm taint nodes ${NODE_NAME} imply.io/agent=true:NoExecute

Then add a matching node selector and toleration for coordinators in your values.yaml:

coordinator:
  nodeSelector:
    example.com/lumi-role: coordinator
  tolerations:
    - key: imply.io/agent
      operator: Exists

The node selector schedules coordinator pods only on nodes labeled example.com/lumi-role=coordinator. The toleration lets them run despite the imply.io/agent taint, which keeps other workloads off the node. Use any label you like, as long as the label on the node and the nodeSelector value match.

Node type requirements​

The following sections describe requirements that apply to specific node types.

Data nodes​

You can use topologySpreadConstraints to spread data pods across failure domains. The settings apply to all data pods as a group. Lumi doesn't support assigning individual data pods to specific zones or failure domains.

Ingestion nodes​

Ingestion tasks need nodes with local disk for storage and enough memory for each task. To run them only on such nodes, label the nodes and add a matching nodeSelector under ingest, for example:

ingest:
  nodeSelector:
    imply.io/server_type: ingest-local-disk-high-mem

You can use any label, as long as the label on the nodes and the nodeSelector value match.

System and infrastructure nodes​

System and infrastructure nodes run the services that support the deployment, such as the gateway, authentication, event collector, update client,and the UI. These services don't have scheduling settings in the Helm values, so they can't tolerate taints. Keep these nodes untainted.

OpenTelemetry collector agent​

The OpenTelemetry collector agent runs as a DaemonSet, meaning one pod runs on every OpenShift node. The agent tolerates the imply.io/agent, imply.io/ingest, and imply.io/is_arm taints with the NoExecute effect. If you taint a node with any other taint, the agent can't run on that node, and the node stops reporting telemetry.

Next step​

After you configure node labels and taints, continue to Install Lumi Enterprise.