Skip to main content

Processors reference

AI summary
Explains how processors in Imply Lumi transform events within pipelines. Covers available processor types for parsing, mapping, redacting, and removing event data. Describes configuration options and best practices for efficient data processing.

About AI summaries.

Pipelines transform incoming events in Imply Lumi. A pipeline is made up of processors that perform discrete transformation tasks, such as creating or redacting a user attribute. This topic describes all supported processors, how to configure them, and examples of how to use them.

To learn how to create a pipeline and add processors, see Manage pipelines and processors.

Processor settings​

Use the following guidelines when configuring processors.

Source and output attributes​

When defining source and output attributes, keep the following in mind:

  • You can use incoming event metadata as source attributes. For example, sourcetype.
  • An attribute created by one processor can be the source attribute for a later processor in the same or different pipeline.
  • Select processors let you use the event message (log body) as the source attribute. For example, the redaction processor.
  • You can write to the event message using the message mapper.
  • You can't use system attributes as source or output attributes.
    Note that system attributes are allowed in the pipeline conditions.

Override output attributes​

Some processors have the option to Override value when output attribute exists. This applies to situations in which you specify an output attribute with the same name as a previously existing user attribute or event metadata. When you choose to override, the processor replaces the original attribute value with the newly processed result.

For example, suppose you send an event with incoming metadata key: value1, and a processor computes the output attribute key: value2. You can choose whether to preserve value1 or override it to value2.

The override applies even when the new value is an empty string or one or more whitespace characters. No processing occurs when the new value is null, such as if you provide a nonexistent source attribute for the attribute mapper.

The grok and regex parsers override any existing user attributes by default. You can't disable override for these processors.

Remove source attribute​

When you map an attribute, the processor doesn't remove the source. To remove it, use the attribute remover.

Removing unused attributes can lead to better query performance and more efficient storage. It also simplifies your search experience and reduces complexity for any data maintenance tasks.

Try out a processor​

For any processor, use the Try it out feature to preview the expected output for your test case. You can also simulate all processors in a pipeline. For details, see Manage pipelines.

Arithmetic processor​

warning

Deprecated. Use the expression processor instead, which covers all arithmetic operations and adds support for functions like ROUND() and string expressions.

Evaluates an arithmetic formula and outputs the result to an attribute.

You can reference existing attributes as variables in the formula. The formula supports the basic operators for addition (+), subtraction (-), multiplication (*), and division (/). Parentheses (()) control the order of operations.

In the arithmetic formula, surround operators with space characters. For example, val1 - val2 is a valid subtraction formula. Without the space characters, the processor interprets val1-val2 as a single attribute.

Configure the processor with the following settings:

  • Override value when output attribute exists: If an attribute with the same name already exists, you can select whether to override its value or leave it unchanged.
  • Round output decimal value: The number of decimal places allowed in the output value. For example, enter 2 to round a result of 0.888 to 0.89.
  • Replace invalid input values with zero: When you select this toggle, Lumi replaces any nonexistent attributes with zero. If you don't select this option, Lumi skips processing and doesn't evaluate the formula.
Example
Processor configuration
Arithmetic formula: (val1 + val2) / (val4 - val3)
Output attribute: computed
Event input
Event metadata: 
val1: 5
val2: 8
val3: 11
val4: 14
Event output
User attribute: computed: 4.333

Attribute mapper​

Maps the value of a source attribute to an output user attribute.

The processor creates a new attribute when it doesn't exist. If an attribute with the same name already exists, you choose to override its value or leave it unchanged.

Example
Processor configuration
Source attribute: status
Output attribute: http_status
Event input
Event metadata: status: 401
Event output
User attribute: http_status: 401

Attribute remover​

Removes one or more source attributes. Specify multiple attributes as a comma-separated list.

Use this processor to drop unneeded fields to reduce storage size and improve query performance. You can also use the attribute remover to drop personally identifiable information, whether to remove it completely or to remove the source metadata after redaction.

Example
Processor configuration
Attributes to remove: userid
Event input
Event metadata: userid: wilma
Event output
User attribute: none

Conditional mapper​

Maps a static value or source attribute to an output user attribute when a user-specified condition passes.

A conditional mapper can take one or more conditions. The processor evaluates them from highest to lowest priority and applies the mapping for the first condition that's met. If no conditions are satisfied, no mapping occurs.

A condition takes the following components:

  • Search expression in Lumi query syntax
  • Value to map, whether a static value or source attribute

In the search expression, you can use the comparison operators =, !=, <, <=, >, >=. The left-hand side of the comparison must refer to an attribute, and the right-hand side must be a string or numeric literal. If the attribute value is null, the result is false. All string comparisons are case-sensitive.

The processor only creates a single output attribute. You can choose whether to override the attribute if it already exists. Create a separate processor for each output user attribute.

Note the following similar processors:

  • A value mapper maps from a static value unconditionally.
  • An attribute mapper maps from a source attribute unconditionally.
  • A lookup mapper evaluates source attributes against a lookup table and creates one or more output attributes.
Example

Consider a static value replacement only for events that have a specific source type.

Processor configuration
Condition: sourcetype=access_combined
Mapper type: Value
Value / Attribute: redacted
Output attribute: user
Event input
Event metadata: 
sourcetype: access_combined
user: wilma
Event output
User attributes: 
sourcetype: access_combined
user: redacted

This configuration only redacts user information when the event satisfies the processor condition for sourcetype and the pipeline condition.

Expression processor​

Computes an arithmetic or string expression and maps the result to an output user attribute. Designate whether you want to override the output attribute if it already exists.

Invalid expressions don't generate an output attribute. For example, if you use a string attribute in an arithmetic formula.

Arithmetic processing​

Perform arithmetic calculations such as val1 + val2. You can use operators for addition (+), subtraction (-), multiplication (*), division (/), and modulo (%). Parentheses (()) control the order of operations.

You can specify the following arithmetic functions:

  • ROUND(n, [precision]): Rounds a number n to the nearest integer.
    Optionally specify precision to round to a specific number of decimal places.
  • FLOOR(n): Rounds a number down to the nearest integer.
  • CEIL(n): Rounds a number up to the nearest integer.

In case an attribute doesn't exist, you can use the COALESCE function to set a default value, such as 0 or 1.

String processing​

Perform string operations such as concatenation: firstName + ' ' + lastName.

The processor converts numeric attributes to strings when they are combined with another attributes that can't be coerced into a number. For example, given the source attributes {"size": 2, "units": "GB"}, the expression size + ' ' + units yields the result 2 GB.

If the source attributes can be coerced to numbers even though they are string values, arithmetic processing applies. For example, with the source attributes {"foo": "2", "bar": "5"}, the expression foo + bar yields the result 7. To you intend to concatenate the values, use CONCAT(foo, bar) to yield the result 25.

You can use the following functions to perform operations such as trimming and text replacement.

  • CONCAT(a, b, ...): Joins two or more inputs into a single value.
  • COALESCE(a, b, ...): Returns the first non-null, non-empty value. Requires at least two arguments.
  • TRIM(s): Removes leading and trailing whitespace from a string.
  • LTRIM(s): Removes leading whitespace from a string.
  • RTRIM(s): Removes trailing whitespace from a string.
  • UPPER(s): Converts a string to uppercase.
  • LOWER(s): Converts a string to lowercase.
  • LENGTH(s): Returns the number of characters in a string.
  • SUBSTRING(s, start, length): Extracts part of a string from zero-based index start for the given length.
  • REPLACE(s, pattern, replacement): For a string s, finds regex pattern and substitutes with replacement.

For more information and examples, see Expression functions reference.

Example
Processor configuration
Expression: raw_bytes/1000 + ' ' + 'kB'
Output attribute: file_size
Event input
Event metadata: raw_bytes: 84352
Event output
User attributes: 
file_size: 84.352 kB
raw_bytes: 84352

Grok parser​

Parses a source attribute into one or more output attributes using a grok expression. You can use the event message as the source attribute.

In the parser configuration, provide the source attribute to parse and a grok expression. A grok expression is made up of one or more grok patterns in the following format:

%{PATTERN_NAME:OUTPUT}

PATTERN_NAME identifies a preset pattern, and OUTPUT is the label you assign to the output value that Lumi stores as a user attribute.

The grok parser extracts structured data when it matches the specified expression, similar to the regex parser. Grok expressions tend to be more human-readable than regex because they use preset templates for common patterns, such as TIMESTAMP_ISO8601. Unlike the regex parser, you don't supply the output attribute names in a separate field; you include them directly in the grok expression.

For a reference on the available patterns, see Grok patterns. Note that you can test your grok patterns using an online parser such as Grok Debugger before you add them to a processor.

Example
Processor configuration
Source attribute: Select the option to Extract from log body
Grok expression: %{TIMESTAMP_ISO8601:time} %{LOGLEVEL:status}: %{GREEDYDATA:message}
Event input
Event message: 2025-08-05 15:45:00 INFO: Starting application...
Event output
User attributes: 
time: 2025-08-05 15:45:00
status: INFO
message: Starting application...

For examples of how to map the extracted values to other event components, see the timestamp mapper and message mapper.

Example with Apache combined log format

This example parses a log in Apache combined log format as represented in the tutorial data.

Processor configuration
Source attribute: Select the option to Extract from log body
Grok expression: 
%{IP:clientip} %{DATA:ident} %{DATA:user} \[%{HTTPDATE:req_time}\] "%{WORD:method} %{DATA:uri} %{DATA:version}" %{NUMBER:status} %{NUMBER:bytes} "%{URI:referer}" "%{GREEDYDATA:useragent}"
Event input
Event message: 
830:1e0e:525:e6a0:6479:cd69:c364:23c3 - - [24/Mar/2025:16:25:29 -0500] "POST /products/23394 HTTP/1.1" 200 1027 "https://techcrunch.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:110.0) Gecko/20100101 Firefox/110.0"
Event output
User attributes: 
bytes: 1027
clientip: 830:1e0e:525:e6a0:6479:cd69:c364:23c3
ident: -
method: POST
version: HTTP/1.1
referer: https://techcrunch.com/
status: 200
req_time: 24/Mar/2025:16:25:29 -0500
uri: /products/23394
user: -
useragent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:110.0) Gecko/20100101 Firefox/110.0

Key-value parser​

Parses key-value pairs from a source attribute into one or more output attributes.

Each key-value pair parsed from the source attribute becomes its own output attribute. For example, the value index=main, source=/var/log/messages, sourcetype=access_combined creates user attributes for index, source, and sourcetype when parsed using the equality pattern.

The processor takes the following configuration:

  • Source attribute: Log body or user attribute

  • Prefix: Optional value to prepend to the output attribute name. When you select a user attribute for the source, the UI pre-populates the attribute name.

    The name of the output attribute takes the format PREFIX.KEY, where KEY is parsed from the source attribute. If the prefix is null, the output attribute name is just the key name. You can use the prefix to store the source name or avoid naming conflicts with other attributes.

    For example, after you parse key-value pairs from a user attribute, you remove the user attribute so that you don't store both an unparsed and parsed version of the same information. With the prefix, you can store request.instanceId instead of just instanceId.

  • Key-value pattern: Pattern that denotes the key-value format

  • Override value when output attribute exists: Whether to overwrite any existing user attributes

The following sections describe supported key-value patterns and any additional required configuration. Unless otherwise specified, the examples don't provide a prefix and override any existing output attributes.

CSV​

Example input: value, where the processor uses the comma delimiter, multiple attributes output, and key header

Parses delimiter-separated values, including CSV and TSV.

Additional configuration:

  • Delimiter: Select comma for CSV, tab for TSV, or custom to provide your own delimiter.
  • Output: Select whether to store parsed results as multiple attributes or a single one.
    • Multiple attributes: Creates a user attribute for each parsed field. For example, a,b,c yields three fields.
      Supply Headers for the respective keys (output attribute names) as a list separated by the same delimiter.
    • Single attribute: Creates a single user attribute that stores all parsed values in an array.
      Supply the name for Output attribute.
Key-value example with CSV parsing, multiple attributes
Processor configuration
Source attribute: Select the option to Extract from user attribute csv_attr
Key-value pattern: CSV
Delimiter: Custom ;
Output: Multiple attributes
Headers: key1;key2
Event input
Event metadata: csv_attr: value1;value2
Event output
User attributes: 
csv_attr: value1;value2
key1: value1
key2: value2
Key-value example with CSV parsing, single attribute
Processor configuration
Source attribute: Select the option to Extract from user attribute csv_attr
Key-value pattern: CSV
Delimiter: Custom ;
Output: Single attribute
Output attribute: key
Event input
Event metadata: csv_attr: value1;value2
Event output
User attributes: 
csv_attr: value1;value2
key: [value1, value2]

Equality​

Example input: key=value

Parses text joined by an equal sign and doesn't have space characters on either side of the sign. If an input contains duplicate keys, such as a=b a=c, the parser only takes the last match: a=c.

Logs can contain key-value pairs separated by whitespace (including tab), comma, semicolon, pipe, and ampersand characters, such as a=b | c=d & e=f. Key-value pairs can also be detected from a multi-line event separated by newlines (\n) or carriage returns (\r). Take note of the following behavior for characters that surround the equal sign:

  • Delimiter within value: key=val1,val2 extracts key: val1
    The parser doesn't extract individual values when the delimiter separates multiple values on a single key.

  • Space after equality: key= val extracts nothing
    The parser doesn't store a key-value pair when a space follows the equal sign.

  • Space within values: key1=Multiword value|key2=Another value extracts key1: Multiword value and key2: Another
    When a non-space delimiter is present, the parser retains the space in the value for non-terminal pairs.

Key-value example with equality parsing
Processor configuration
Source attribute: Select the option to Extract from log body
Key-value pattern: Equality
Event input
Event message: index=main, source=/var/log/messages, sourcetype=access_combined
Event output
User attributes: 
index: main
sourcetype: access_combined
source: /var/log/messages

JSON​

Example input: {"key": "value"}

Parses key-value pairs from JSON objects.

Select the checkbox Flatten into a single level to flatten the JSON structure, else preserve original nesting. For example, when parsing the input value {"key1": {"key2": "value"}}, you can store "key1": {"key2": "value"} (unchecked) or key1.key2: value (checked).

Key-value example with JSON parsing
Processor configuration
Source attribute: Select the option to Extract from user attribute request
Key-value pattern: JSON
Flatten into a single level: true
Subsequent processor
Attribute remover to remove request
Event input
Event metadata: request: {"customerId": "abc123", "device": {"id": "i714"}}
Event output
User attributes: 
customerId: abc123
device.id: i714
Key-value example with JSON parsing, with prefix
Processor configuration
Source attribute: Select the option to Extract from user attribute request
Prefix: request
Key-value pattern: JSON
Flatten into a single level: true
Subsequent processor
Attribute remover to remove request
Event input
Event metadata: request: {"customerId": "abc123", "device": {"id": "i714"}}
Event output
User attributes: 
request.customerId: abc123
request.device.id: i714

Regex​

Example input: key_value with regex (\w*)_(\w*)

Parses text that adheres to a regular expression with two capture groups, where the first group is the key and the second group is the value. To only extract values and provide your own list of key names, See regex parser.

Select the checkbox Combine values from duplicate keys to capture multiple values for a recurring key. Otherwise, the processor only stores the first match. For example, when parsing the value key:val1 key:val2 key:val3 with the regex (\w*):(\w*), you can store key:val1 (unchecked) or key:val1|val2|val3 (checked).

Key-value example with regex parsing
Processor configuration
Source attribute: Select the option to Extract from log body
Key-value pattern: Regex
Regular expression: (\w+): ((?:.(?!\w+:))+)
Event input
Event message: 
Apr 23 08:45:11 192.168.56.1 Apr 23 08:45:10 HERMES/192.168.56.22 THOR: Info: MODULE: UserAccounts USER: neo PRIV: 0 LAST_LOGON: 04/22/2026 17:33:01 BADPWCOUNT: 2 NUM_LOGONS: 341 PASS_AGE: 12.00 days
Event output
User attributes: 
LAST_LOGON: 04/22/2026
MODULE: UserAccounts
PASS_AGE: 12.00 days
NUM_LOGONS: 341
PRIV: 0
USER: neo
BADPWCOUNT: 2

XML​

Example input: <root><key>value</key></root>

Parses text within an XML document's root element. Each input requires a single root element. The key-value parser doesn't extract the root name.

Select the checkbox Flatten into a single level to flatten the XML structure, else preserve original nesting.

Key-value example with XML parsing
Processor configuration
Source attribute: Select the option to Extract from log body
Key-value pattern: XML

Event input
Event message:

<guestbook><guestcount>123</guestcount><guest rsvp="true">Wilma Rudolph</guest><venue><reception>courtyard</reception></venue></guestbook>

Event output with flattening
User attributes:

guest: Wilma Rudolph
guestcount: 123
venue.reception: courtyard
guest.rsvp: true

Event output retaining nested structure
User attributes:

guest: {
  rsvp: true
  Wilma Rudolph
}
guestcount: 123
venue: {
  reception: courtyard
}

Lookup mapper​

Looks up source attributes in a user-provided lookup table, and creates output attributes from the table values.

A lookup table contains a delimiter-separated set of values. You can provide the column headers as part of the table itself or as comma-separated values in the Headers field. Denote the type of character separation in Delimiter. Be sure to match the delimiter to your lookup table. For example, the delimiter comma , is different from comma with a space , .

In Source attributes, enter the incoming attributes to use as lookup IDs from the table. Each source attribute should correspond to a table column.

In Output attributes, enter the attributes to create from the lookup. Each output attribute should correspond to a table column. Lumi assigns the attribute value from the first matched row. If your events already contain the output attributes, you can designate whether to override existing values.

Consider an example with the following lookup table:

product_idcategorydescription
23394FurnitureLeather Sectional Sofa
32729ElectronicsRaspberry Pi 5
23002BooksMan's Search for Meaning
23394InstrumentsAnalog Theremin
78905JewelryArt Deco Diamond Bracelet

You specify product_id as the source attribute and category, description as the output attributes. Based on the incoming event, the processor does the following:

  • When a source attribute contains product_id: 23002, the processor creates the output attributes category: Books and description: Man's Search for Meaning.
  • When a source attribute contains product_id: 1234, the processor doesn't create any attributes.
Example

This example adds the description user attribute for events that store a specific product ID and category.

Processor configuration
Headers: Lookup CSV includes header line
Lookup CSV: 
product_id,category,description
23394,Furniture,Leather Sectional Sofa
32729,Electronics,Raspberry Pi 5
23002,Books,Man's Search for Meaning
23394,Instruments,Analog Theremin
28201,Jewelry,Art Deco Diamond Bracelet
Delimiter: ,
Source attributes: product_id,category
Output attribute: description
Event input
Event metadata: 
product_id: 23394
category: Instruments
Event output
User attributes: 
product_id: 23394
category: Instruments
description: Analog Theremin

Note that if you only select product_id as the source attribute, the resulting user attribute would be description: Leather Sectional Sofa, since it's the first matched row for product ID 23394.

Message mapper​

Maps the value of a source attribute to the event message.

You have the option to overwrite the event message with an empty string when the source attribute is missing or empty.

Example
Preceding processor
Grok parser to extract message: Starting application...
Processor configuration
Source attribute: message
Event input
Event message: 2025-08-05 15:45:00 INFO: Starting application...
Event output
Event message: Starting application...

Redaction processor​

Redacts a source attribute using a regular expression. You can use the event message as the source attribute.

The processor overrides the source attribute to store the redacted content. You can search on the redacted content such as user!=*redacted*.

Take note of the following for regex:

  • Must have at least one capture group.
  • Can match zero or more times in the source attribute. Each match is redacted.
  • Captures the entire value when the pattern is (.+). However, if you're redacting the entire value, consider the value mapper with override enabled.

In the processor configuration, you specify the redaction strategy, which determines how Lumi identifies and performs the replacement. The following sections describe these strategies in more detail.

String redaction​

Replaces the entirety of every regex match. Capture groups let you optionally retain content from the match.

To keep a capture group, backreference it in the replacement text using the syntax $N, where N is the one-based index of the group. For example, $2 references the second capture group. You can specify the capture groups in any order, for example, $3-$1-$2. When the replacement text doesn't reference a capture group, any specified capture group is ignored.

Consider the email address username@example.org that you want to redact to u***@example.org. The following configuration generates the redacted output:

  • Regex: (\w)\w*(@\w+\.\w+)
    \w matches word characters [a-zA-Z0-9_]. The first capture group matches the first character u, and the second capture group matches the email domain @example.org.
  • Replacement text: $1***$2
    The replacement text retains the two capture groups with asterisk characters in between.

See the following for additional examples.

Example of string redaction for a user identification number

This example redacts a Social Security number and retains the last four digits.

Processor configuration
Source attribute: Select the option to Extract from log body
Regular expression: (\d{3})-(\d{2})-(\d{4})
Strategy: String
Replacement text: xxx-xx-$3
Event input
Event message: 
2023-10-27 10:01:05 INFO UserID: 88421 - Username: jdoe - SSN: 999-00-1111 - Action: UpdateRecord
Event output
Event message: 
2023-10-27 10:01:05 INFO UserID: 88421 - Username: jdoe - SSN: xxx-xx-1111 - Action: UpdateRecord
Example of string redaction on multiple matches

When there are multiple regex matches against the source attribute, the processor operates on each match. Consider a log that contains two phone numbers.

Processor configuration
Source attribute: Select the option to Extract from log body
Regular expression: (phone=)"\d{3}-\d{3}-\d{4}"
Strategy: String
Replacement text: $1[REDACTED]
Event input
Event message: 
user=wilma, phone="800-555-0100", phone="800-555-0100"
Event output
Event message: 
user=wilma, phone=[REDACTED], phone=[REDACTED]

Hash redaction​

Replaces only the capture group for every regex match. The purpose of the capture group is to define the content to redact.

The processor replaces each capture group with its own cryptographic hash. Note the difference from the string strategy, where the entire regex is replaced and not just the capture groups. You can't backreference capture groups in the hash strategy.

In the processor configuration, select a hash algorithm such as MD5 or SHA-256. You can also supply a salt, such as a random string, to combine with the source attribute before hashing. Select whether to prepend or append the salt to the source attribute.

Example of hash redaction of a password

This example redacts the password from a log and replaces it with a hash.

Processor configuration
Source attribute: Select the option to Extract from log body
Regular expression: password=(.+);
Strategy: Hash
Algorithm: SHA-512
Salt: True
    Value: jv4w7m
    Position: Append
Event input
Event message: 
connection=db;user=admin;password=secret123;host=local
Event output
Event message: 
connection=db;user=admin;password=c8a776ac50189ec7ad12c9573865717cad8b37cba9af872059094fed920e70a642ac5d84e63a36da0c8fd6b94da9b0e6fdee07f12b42352afd1304155763d13b;host=local

Regex parser​

Parses a source attribute into one or more output attributes using a regular expression. You can use the event message as the source attribute.

The processor takes the following configuration:

  • Source attribute: Log body or user attribute

  • Regular expression: Regex with a capture group for each output attribute value. The number of capture groups, denoted by (), determines the number of output attributes. To test regular expressions, try out the processor, or use a free regex parser such as Regex101.

    If a capture group matches more than one result, the processor only takes the first match. For example, for the input text hello world, the pattern (\w+) matches hello and world separately, but the processor only stores hello.

  • Output attributes: Comma-separated list of output attribute names. The number of names must match the number of regex capture groups. If the source already contains the attribute names, you can also use the key-value parser with the regex option to extract both attribute names and values.

    If you have an existing user attribute with the same name as one of the specified output attributes, the parser overwrites the previously existing user attribute. This behavior applies even when the match is an empty string or whitespace character.

Example
Processor configuration
Source attribute: Select the option to Extract from log body
Regular expression: status: \[(\w*)\]
Output attributes: status
Event input
Event message: Deployment successful. System 1 status: [ok] System 2 status: [alert]
Event output
User attribute: status: ok
Example with Apache combined log format

This example parses a log in Apache combined log format as represented in the tutorial data.

Processor configuration
Source attribute: Select the option to Extract from log body
Regular expression: 
([^ ]*) ([^ ]*) ([^ ]*) \[([^\]]*)\] "(\S+)(?: +([^\"]*?)(?: +(\S+))?)?" ([^ ]*) ([^ ]*)(?: "([^\"]*)" "([^\"]*)")?
Output attributes: 
clientip, ident, user, req_time, method, uri, version, status, bytes, referer, useragent
Event input
Event message: 
830:1e0e:525:e6a0:6479:cd69:c364:23c3 - - [24/Mar/2025:16:25:29 -0500] "POST /products/23394 HTTP/1.1" 200 1027 "https://techcrunch.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:110.0) Gecko/20100101 Firefox/110.0"
Event output
User attributes: 
bytes: 1027
clientip: 830:1e0e:525:e6a0:6479:cd69:c364:23c3
ident: -
method: POST
version: HTTP/1.1
referer: https://techcrunch.com/
status: 200
req_time: 24/Mar/2025:16:25:29 -0500
uri: /products/23394
user: -
useragent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:110.0) Gecko/20100101 Firefox/110.0

Status mapper​

Maps the value of a source attribute to the event status.

Lumi attempts to map status codes to human-readable values. For example, the HTTP status code 500 maps to Error. For more information, see the system attribute for status.

You can optionally include a fallback value that Lumi sets for the status when the source attribute doesn't exist or if it's unable to be interpreted.

Example
Processor configuration
Source attribute: http_code
Event input
Event metadata: http_code: 200
Event output
System attribute: status: ok

Note that status here is a system attribute, not a user attribute. If you want to remove the source attribute after the status mapping, use the attribute remover.

Timestamp mapper​

Maps the value of a source attribute to the event timestamp. Provide the name of the source attribute, timestamp format, and the time zone (optional).

Supported timestamp formats include ISO 8601, Common Log Format, and Unix epoch values. To automatically detect the format, select Auto. To define your own timestamp pattern, select Custom, and enter your pattern. Custom formats use Java DateTimeFormatter syntax. For details and examples, see Time formats.

When the timestamp is embedded in the event message, you can map it to the event timestamp as follows:

  1. Create a processor, such as the regex parser, to extract the timestamp as a new attribute.
  2. Use the newly extracted attribute in the timestamp mapper.
  3. Clean up the extracted attribute using the attribute remover.

For more details, see Manual timestamp mapping.

Example
Event input
Event message: 2025-08-05 15:45:00 INFO: Starting application...
Initial parsing
Grok parser to extract time: 2025-08-05 15:45:00
Timestamp processor
Source attribute: time
Time format: Custom: yyyy-MM-dd HH:mm:ss
Time zone ID: supply your time zone
Event output
Event timestamp: Aug 05, 03:45:00.000 PM
Example with Apache combined log format
Event input
Event message: 29.182.147.96 - - [24/Mar/2025:16:25:29 -0500] "POST /products/23394 ...
Initial parsing
Regex parser to extract time: 24/Mar/2025:16:25:29 -0500
Timestamp processor
Source attribute: time
Time format: CLF (Common Log Format)
Time zone ID: leave empty
Event output
Event timestamp, viewed from PDT time: Mar 24, 02:25:29.000 PM

In this example, the event message recorded the time as 4:25 PM CDT (denoted by the -0500 time zone specification). The user observed the event from the America/Los_Angeles time zone (PDT). As a result, the event displays the timestamp in Lumi as two hours prior.

Value mapper​

Maps a static value to an output user attribute.

For the static value, you can enter your own value or assign the Unix timestamp of event indexing. The Unix time represents seconds from Unix epoch—January 1, 1970, at 00:00:00 UTC. For more information about the event indexing timestamp, see Timestamp handling.

The processor creates a new attribute when it doesn't exist. If an attribute with the same name already exists, you can choose to override its value or leave it unchanged.

Example
Processor configuration
Static value: example.com
Event input
Event metadata: host: 23.192.228.84
Event output
User attribute: host: example.com

Limitations​

Lumi doesn't currently support extractions on time fields.

Learn more​

See the following topics for more information: