Skip to main content

Work with predefined pipelines

Predefined pipelines are built-in pipelines that Imply Lumi provides for processing specific types of data. They apply standard parsing and enrichment rules so your data is structured and ready to use as soon as it’s ingested. This saves you from creating custom processors for common data sources.

You can use predefined pipelines as they are, or customize them to fit your requirements.

In this topic, you’ll learn how to add and manage predefined pipelines.

Add a predefined pipeline​

To add a predefined pipeline:

  1. In the Lumi navigation menu, click Pipelines.
  2. Click Predefined pipelines.
  3. Select a predefined pipeline and click Add pipeline. The pipeline appears at the bottom of the list.
  4. Click the ellipsis next to the pipeline and select Edit. You can:
    • Edit the name and description.
    • Update the pipeline condition if you want to change how Lumi filters events for the pipeline. See Pipeline conditions for more information.
  5. Use the ellipsis menu to reorder, duplicate, or disable the pipeline.

Click the arrow next to the pipeline name to expand it and view its processors. To edit the processors, see Customize processors in a predefined pipeline.

Customize a predefined pipeline​

To customize a predefined pipeline, first add the pipeline and then duplicate it. You can then make changes to the copy.

You might customize processors to add or remove fields. You might also create two versions of the same pipeline with different conditions to handle similar events for different use cases.

To customize processors in a pipeline:

  1. Add a predefined pipeline and create a duplicate. The pipeline copy appears in the list with (Copy) appended to the name. The original predefined pipeline is marked with a box icon.
  2. Click the ellipsis next to the pipeline copy and edit its details as required.
  3. Customize the pipeline:
    • To add processors, click the ellipsis and select Add processors.
      See Create a processor for more information.
    • To edit processors, expand the pipeline and click the ellipsis next to a processor. Reorder, edit, duplicate, disable, or delete a processor.

The following example shows a VPC Flow Log Processing pipeline and a duplicate:

Predefined and custom pipelines

Available predefined pipelines​

Lumi includes the following predefined pipelines:

Learn more​

See the following topics for more information: