Skip to main content

S2S event parsing reference

AI summary
Explains how Imply Lumi parses events using Splunk® properties for line breaking and timestamp extraction. Applies to S2S protocol integrations `tcpout` and `httpout` and the Splunk HEC integration with the `/raw` endpoint.

About AI summaries.

Imply Lumi supports a subset of Splunk® properties to configure how to parse incoming events in the S2S protocol.

You assign these properties as S2S attributes on the IAM key for the following integrations:

If your data isn't compatible—or can't be made compatible—with the available options, see other approaches to send events and explore how you can use pipelines to transform incoming events.

Event line breaking​

Lumi determines event breaks using the following heuristics:

  1. Detect and separate lines by newline characters \r or \n.
  2. Detect a datetime field as the start of a new event.

When an event spans multiple lines, Lumi merges them before storing the event. A datetime field at the start of a line identifies where the next event begins. To disable merging of multi-line events, clear the Should line merge checkbox. This option corresponds to SHOULD_LINEMERGE in a Splunk props.conf configuration file. For more information, see the Splunk documentation for event line breaking.

Event timestamps​

Lumi can automatically parse and extract timestamps from incoming events. For details about timestamp detection and assignment in Lumi, see Timestamp detection.

In cases where Lumi can't detect timestamps, you can configure timestamp parsing using the properties in this section. They correspond to properties of the same name in a Splunk props.conf file. For more information, see the Splunk documentation for timestamp recognition.

Time prefix​

String regular expression that matches the text pattern preceding the timestamp.

  • Default: empty string
  • Example: [\w\.:]*\s[\w-]*\s[\w-]*\s\[
  • Splunk configuration: TIME_PREFIX

Max timestamp lookahead​

Integer number that indicates the maximum character position to look for a timestamp. If you also set the time prefix, this position starts after the matched text.

  • Default: 128
  • Example: 20 means to search for the timestamp in the first 20 characters of the event message, following any time prefix.
  • Splunk configuration: MAX_TIMESTAMP_LOOKAHEAD

Time format​

String pattern in strptime format to extract timestamps. Supports Splunk-specific time variables. See the SPL reference for details.

  • Default: empty string
  • Example: %Y-%m-%d %H:%M:%S
  • Splunk configuration: TIME_FORMAT

Example​

Consider an event with the following message:

830:1e0e:525:e6a0:6479:cd69:c364:23c3 - - [24/Mar/2025:16:25:29 -0500] "POST /products/23394 HTTP/1.1" 200 1027 "https://techcrunch.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:110.0) Gecko/20100101 Firefox/110.0"

The timestamp, 24/Mar/2025:16:25:29 -0500, appears after the client IP address, any identity provided for the client, and any provided user ID.

The following settings control how Lumi parses this timestamp:

  1. Time prefix: [\w\.:]*\s[\w-]*\s[\w-]*\s\[
    Look for the timestamp following this regular expression pattern:

    • [\w\.:]* to match zero or more word characters, periods, or colons
    • \s to match a single space
    • [\w-]* to match zero or more word characters or hyphens
    • \s to match a single space
    • \[ to match a open square bracket

    Matches the string 830:1e0e:525:e6a0:6479:cd69:c364:23c3 - - [.

  2. Max timestamp lookahead: 20
    Look for the timestamp within the next 20 characters.
    Starts from 24, the characters immediately following the prefix.
    Matches the value 24/Mar/2025:16:25:29.

  3. Time format: %d/%b/%Y:%H:%M:%S
    Format the discovered timestamp according to this pattern.

Learn more​

For more information, see the following topics: