Skip to main content

Enrich Lumi events with Splunk lookups

Lookups in Splunk® let you enrich search results with data from external sources, such as a CSV file that maps usernames to roles or IP addresses to locations. This topic explains how to extend an existing Splunk lookup to enrich Imply Lumi events through transparent federated search.

Workflow​

Lumi supports lookups through Knowledge Bundle Replication (KBR), which propagates lookup definitions and CSV files from Splunk to Lumi. Lumi performs the enrichment and returns enriched results to Splunk.

The following workflow describes the process. It assumes that a Splunk admin has configured a lookup table and lookup definition in Splunk.

  1. A user runs a query in Splunk that includes a lookup command.
  2. The transparent federated provider routes the query to Lumi.
  3. Lumi performs the lookup enrichment using the data propagated from one or more CSV files and returns enriched events to Splunk.
  4. In Splunk, the user sees results combined from both Splunk and Lumi.
Lookup query process flow

Example workflow​

In this example, a security analyst wants to investigate web traffic from high risk users. The Lumi events contain web access logs with a user field. A lookup table (user_lookup.csv) is uploaded and associated with the user_lookup definition in Splunk. It maps usernames to role, department, and risk_level.

The analyst runs the following query in Splunk:

index=main earliest=-3d
| lookup user_lookup user OUTPUT role, department, risk_level 
| where risk_level="high"
| table user, role, department, uri, host, status

Lumi receives the query through the transparent federated provider, performs the lookup enrichment using the data propagated from user_lookup.csv, and returns enriched events to Splunk. The analyst sees only events where risk_level is high, with full user context from the lookup.

Example results:

Lookup example results

Splunk requirements​

This section details the Splunk configurations required to use lookups against Lumi.

Transparent mode​

Lookups are only supported with transparent federated search. They don't work with standard mode federated search.

Lookup permissions​

You must set permissions on both the lookup table file and the lookup definition in Splunk to one of the following:

  • This app only
  • All apps (system)

If either is set to Keep private, the lookup won't work with federated search.

If you reference the CSV file directly by filename in the lookup command (for example, lookup user_lookup.csv user OUTPUT role), only the lookup table file permissions apply. You don't need a lookup definition in this case.

Note that Lumi doesn't enforce Splunk's role-based permissions on lookups. A lookup set to This app only is queryable by any user via any Splunk app, regardless of their Splunk role. Keep this in mind when you set permissions.

Using lookups with data models​

You can combine a lookup with a data model query. When you do, you must specify the data model field names in the lookup command. Lumi uses the IAM key mapping to translate field names to data model names before running the lookup, but doesn't automatically apply those translations to the lookup command. If you use Lumi-native field names in the lookup command, the lookup won't find them.

For example:

| tstats count FROM datamodel=Web.Web WHERE Web.status=500 by Web.user
| lookup user_lookup user AS Web.user OUTPUT role, department, risk_level

Lumi requirements​

This section details the Lumi configurations to ensure lookup queries return complete results.

Supported lookup types​

Lumi supports CSV lookups.

You can also use the iplocation command to extract location information from IP addresses using the geolocation database supported by Splunk. See iplocation in the Splunk documentation for more information.

Field availability​

The fields you're matching on must exist in your Lumi events. KBR propagates the lookup definition and CSV file to Lumi, but doesn't create or extract fields. If the match field isn't present in your Lumi events, the lookup returns no results. If the field doesn't exist in your Lumi events, you can use a pipeline to extract or rename it at index time.

Learn more​