How to send events with S2S

In this tutorial, you learn how to send data to Imply Lumi using Splunk® and the Splunk-to-Splunk (S2S) TCP-based data transmission protocol. This tutorial uses the tcpout integration, but you can also follow it with the httpout integration.

The steps show you how to:

• Configure your Splunk instance to access the TCP endpoint in Lumi.
• Apply settings to parse events.
• Send a log file to Lumi.
• Preview the data and view events in Lumi.

To complete the steps, you use sample web traffic data from a fictional online store. For details about the dataset and its format, see the tutorial data overview.

The following diagram summarizes the end-to-end process of sending events to Lumi using S2S. Yellow shaded boxes represent steps taken within Lumi, and blue shaded boxes represent steps taken outside Lumi. Click any box in the diagram to jump to that step.

Prerequisites

To complete the tutorial, you need the following:

• Access to Lumi with the Data manager role or higher.
  For information on roles and permissions, see Manage roles.
• A Splunk universal or heavy forwarder.
  For information on the Splunk forwarders, refer to the Splunk documentation.
• Port 9997 open for outbound TCP traffic on the machine forwarding the events.

A basic understanding of event forwarding in Splunk is helpful but not required to complete this tutorial.

1. Create an IAM key

In this section, you create an IAM key and set event parsing attributes on the key.

1. From the Lumi navigation menu, click Integrations > tcpout.

   

2. In the Select or create an IAM key pane, click Select or create key > Create key.

3. Enter the following information in the Create IAM key dialog:

   • General:
     • Name: tutorial-s2s
       Only the Name field is required to create the IAM key.
     • Description: IAM key for S2S tutorial
   • Global attributes: Events sent to Lumi contain env and team as system attributes.
     • Environment: tutorial
     • Team: learning
   • S2S attributes: These settings apply to the sample events for this tutorial. The S2S attributes aren't stored with the events. For more details on the attributes and these example values, see Event parsing for S2S.
     • Time prefix: [\w\.:]*\s[\w-]*\s[\w-]*\s\[
       Regex pattern representing the string that precedes the timestamp.
     • Max timestamp lookahead: 20
       Number of characters into the line in which to search for the timestamp.
     • Time format: %d/%b/%Y:%H:%M:%S
       Format of the timestamp.

   

4. Click Create to add the key and enable the integration for it.

5. In the Configure event forwarding section, view the configuration for Splunk outputs.conf. This configuration is specific to your IAM key and Lumi environment. You copy this configuration into your Splunk file in the next section.

   

2. Add the Lumi receiver in Splunk outputs

In this section, you add Lumi as a receiver on the Splunk forwarder. You edit the Splunk configuration file outputs.conf to create a tcpout processor with Lumi details. See the Splunk documentation for additional information on configuring receivers in Splunk.

1. In your terminal, navigate to the directory that contains the CLI of your Splunk installation: $SPLUNK_HOME on Unix-based machines or %SPLUNK_HOME% on Windows.

   info

   The SPLUNK_HOME variable is set at installation time and specifies the path where Splunk is installed. For example, /Applications/SplunkForwarder on a Mac OS or C:\Program Files\Splunk on Windows.

2. In a text editor, open a new file called outputs.conf inside the directory etc/system/local/.

3. Copy and paste the Lumi receiver configuration into the file. Your file should resemble the following:

   [tcpout]
   defaultGroup = logs_lumi
   
   [tcpout:logs_lumi]
   server = LUMI_ENDPOINT
   token = IAM_KEY_TOKEN
   useSSL = true
   useClientSSLCompression = false

   Ensure your values for LUMI_ENDPOINT and IAM_KEY_TOKEN represent your specific endpoint and token. For details on these settings, see Send events with S2S.

4. Save and close the file. You restart Splunk in a later step, and Splunk encrypts the token value upon restart.

3. Define processing rules in Splunk props

In this section, you define processing rules for the Splunk forwarder in the configuration file props.conf. See the Splunk documentation for additional information on processing rules in Splunk.

1. In a text editor, open a new file called props.conf inside the directory etc/system/local/.

2. Copy and paste the following content into the file:

   TIME_PREFIX = [\w\.:]*\s[\w-]*\s[\w-]*\s\[
   MAX_TIMESTAMP_LOOKAHEAD = 20
   TIME_FORMAT = %d/%b/%Y:%H:%M:%S
   SHOULD_LINEMERGE = false

   The first three properties describe timestamp extraction. These values match the timestamp parsing properties you assigned on the IAM key.

   The rule SHOULD_LINEMERGE = false treats each line of data as a separate event. By default, Splunk combines several lines of data into a single multi-line event.

3. Save and close the file.

4. Specify the source of events in Splunk inputs

In this section, you direct the forwarder to an input source of log data. You edit the Splunk configuration file inputs.conf to create a batch input with the log file details. See the Splunk documentation for additional information on input sources in Splunk.

1. In a text editor, open a new file called inputs.conf inside the directory etc/system/local/.

2. Copy and paste the following content into the file:

   sourcetype = access_combined
   
   [batch:///Applications/SplunkForwarder/site_visitors.log]
   move_policy = sinkhole
   
   [blacklist:/Applications/SplunkForwarder/var/log]

   Note the following stanzas in this configuration:

   • You set the sourcetype of the file to the recognized type access_combined. Refer to the Splunk documentation for a list of source types that Splunk automatically recognizes.

   • You use the batch input to effectively upload a file through the forwarder. This is a destructive method to send events; Splunk deletes the file after reading it. If you have a file you want to continuously monitor, use the monitor input instead.

   • With the default forwarder settings, the forwarder sends internal Splunk log files such as metrics.log. The blacklist stanza in this example prevents those log files from being sent. If you prefer to monitor these files, remove the blacklist stanza. You can filter specific source files in the explore view.

3. Start the Splunk forwarder:

   ./bin/splunk start

   If the forwarder is already running, be sure to restart it: ./bin/splunk restart.

4. To send the data, download the example data file site_visitors.log and save it in your SPLUNK_HOME directory. Splunk automatically reads the data and removes the file.

5. Preview data

In this section, you preview the data you sent to Lumi and view the events in the explore view.

1. In Lumi, return to the tcpout integration page. In the Preview incoming data pane, view the newly added events.

   

2. Select Explore events. Lumi takes you to the explore view and applies search filters for your IAM key and the integration. You may need to edit the time range or refresh the page to see the events.

   

3. Select an event to view the event and its attributes. For details on the attributes, see Event model.

   

For information on searching events and filtering on event attributes, see Search events with Lumi.

Learn more

For more information, see the following topics:

• Send events with S2S for details on the Splunk-to-Splunk integration.
• Event parsing for S2S for reference on parsing events using the S2S protocol.
• Send events to Lumi for other options to send events.