Skip to main content

Event parsing for S2S events

AI summary
Describes supported timestamp extraction patterns for Imply Lumi when ingesting events via the Splunk-to-Splunk® protocol, including timestamp configuration properties compatible with Splunk's props.conf settings.

About AI summaries.

This topic lists the supported settings in Imply Lumi to parse events using the Splunk®-to-Splunk (S2S) protocol. Both S2S integrations, tcpout and httpout, use this protocol.

Configure these settings on the IAM key you use to send events to Lumi. Add the S2S integration to the key in order to set these options. For details about creating an IAM key, see Manage IAM keys.

Lumi supports a subset of Splunk properties to configure event parsing. If your data isn't compatible—or can't be made compatible—with the available options, see other approaches to send events to Lumi.

Event line breaking​

Lumi determines event breaks using the following heuristics:

  1. Detect and separate lines by newline characters \r or \n.
  2. Detect a datetime field as the start of a new event.

When an event spans multiple lines, Lumi merges them before storing the event. A datetime field at the start of a line identifies where the next event begins. To disable merging of multi-line events, clear the Should line merge checkbox. This option corresponds to SHOULD_LINEMERGE in a Splunk props.conf configuration file. For more information, see the Splunk documentation for event line breaking.

Event timestamps​

Lumi can automatically parse and extract timestamps from incoming events. For details about timestamp detection and assignment in Lumi, see Timestamp handling.

In cases where Lumi can't detect timestamps, you can configure timestamp parsing using the properties in this section. They correspond to properties of the same name in a Splunk props.conf file. For more information, see the Splunk documentation for timestamp recognition.

Time prefix​

String regular expression that matches the text pattern preceding the timestamp.

  • Default: empty string
  • Example: [\w\.:]*\s[\w-]*\s[\w-]*\s\[
  • Splunk configuration: TIME_PREFIX

Max timestamp lookahead​

Integer number that indicates the maximum character position to look for a timestamp. If you also set the time prefix, this position starts after the matched text.

  • Default: 128
  • Example: A value of 100 tells Lumi to search for the timestamp within the first 100 characters of the event.
  • Splunk configuration: MAX_TIMESTAMP_LOOKAHEAD

Time format​

String pattern in strptime format to extract timestamps.

  • Default: empty string
  • Example: %Y-%m-%d %H:%M:%S
  • Splunk configuration: TIME_FORMAT

Example​

Consider an event with the following message:

830:1e0e:525:e6a0:6479:cd69:c364:23c3 - - [24/Mar/2025:16:25:29 -0500] "POST /products/23394 HTTP/1.1" 200 1027 "https://techcrunch.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:110.0) Gecko/20100101 Firefox/110.0"

The timestamp, 24/Mar/2025:16:25:29 -0500, appears after the client IP address, any identity provided for the client, and any provided user ID.

The following settings control how Lumi parses this timestamp:

  1. Time prefix: [\w\.:]*\s[\w-]*\s[\w-]*\s\[
    Look for the timestamp following this regular expression pattern:

    • [\w\.:]* to match zero or more word characters, periods, or colons
    • \s to match a single space
    • [\w-]* to match zero or more word characters or hyphens
    • \s to match a single space
    • \[ to match a open square bracket

    Matches the string 830:1e0e:525:e6a0:6479:cd69:c364:23c3 - - [.

  2. Max timestamp lookahead: 20
    Look for the timestamp within the next 20 characters.
    Starts from 2, the position immediately following the prefix.
    Matches the value 24/Mar/2025:16:25:29.

  3. Time format: %d/%b/%Y:%H:%M:%S
    Format the discovered timestamp according to this pattern.

Learn more​

For more information, see the following topics: