Skip to main content

Lumi Enterprise

Imply Lumi Enterprise is an Imply Lumi deployment that lives in your AWS environment, keeping your data within your infrastructure. If you have strict data sovereignty, security, and compliance requirements around logs, Lumi Enterprise can be a better fit than the Imply-hosted Lumi Cloud.

Imply provides a Terraform module that simplifies Lumi Enterprise deployment. The module handles all the deployment steps for Lumi Enterprise, including the AWS infrastructure and dependent PaaS Services, such as Imply's distribution of Apache® Druid, Amazon Managed Streaming for Apache® Kafka (Amazon MSK), Amazon Elastic Kubernetes Service (Amazon EKS), Amazon S3, and Amazon RDS Aurora MySQL. You don't need to perform any additional installation or configuration outside of the Terraform module.

Lumi uses Imply’s distribution of Druid to power its services and MSK to stream event data from various sources into Lumi.

Lumi​

The following table provides a basic overview of the Lumi nodes you deploy:

Lumi nodesDescription
CoordinatorThe Lumi Coordinator manages where to store data and how long its retained in Lumi. Additionally, it manages tasks like ingestion.
BrokerThe Lumi Broker routes your Lumi queries. These are unrelated to MSK brokers.
DataLumi data nodes store data on two tiers, a hot tier and a virtual tier.
IngestionMSK streams event data to the ingestion nodes in your Lumi deployment where the Indexer service performs the ingestion.

Event data ingestion​

MSK streams your event data to Lumi. The Indexer service on the ingestion nodes run the tasks. After the data is ingested, Lumi stores it on data nodes.

Event data storage​

Any data that you ingest into Lumi is stored and managed as segments that are based on intervals of time. These segments are stored on Lumi data nodes.

Lumi supports two types of data nodes:

  • Hot tier data nodes store data that you want to query frequently.
  • Virtual tier data nodes store data that you query irregularly. When needed, data stored in the virtual tier gets loaded.

On both kinds of Lumi data nodes, Historicals store your queryable data. As part of the segment loading process, Lumi indexes and pre-aggregates data to help speed up queries.

Once loaded, the segments are queryable through the Broker service. The Lumi Broker routes your queries to the data node containing the relevant time-interval segment. This segment-based storage is one of the features that helps Lumi efficiently store and retrieve data you query. Only data that falls within the time interval for your query is accessed during your query.

If you query data that's streaming to Lumi and hasn't been loaded onto a Historical yet, the Broker routes your query to the Indexer.

Lumi Management Console​

After you deploy Lumi Enterprise, you can monitor your deployment through the Lumi Management Console. The console is hosted by Imply.

As you monitor your deployment, use the Terraform module to scale your cluster to better meet your needs.

For more information, see Lumi Management Console.

Get started​

When you're ready to get started, learn how to install Lumi Enterprise using Terraform.