Skip to main content

How to send events with an S3 ingest action

In this tutorial, you learn how to send events to Imply Lumi using a Splunk® S3 ingest action.

The steps show you how to:

  • Configure an S3 ingest action to route events to Lumi.
  • Send events from a sample log file to Lumi through the S3 ingest action.
  • Preview the data and view events in Lumi.

To complete the steps, you use sample web traffic data from a fictional online store. For details about the dataset and its format, see the tutorial data overview.

The following diagram summarizes the end-to-end process of sending events to Lumi using Splunk S3. Yellow shaded boxes represent steps taken within Lumi, and blue shaded boxes represent steps taken outside Lumi. Click any box in the diagram to jump to that step.

Prerequisites​

To complete the tutorial, you need the following:

  • Access to Lumi with the Data manager role or higher.
    For information on roles and permissions, see Manage roles.
  • A Splunk heavy forwarder configured in your environment for routing events to an S3 destination.
    For more information, see Use forwarders to get data into Splunk Enterprise.
  • Port 443 open for outbound TCP traffic on the machine running the Splunk heavy forwarder.

1. Create an IAM key​

In this section, you create an IAM key and set event parsing attributes on the key.

  1. From the Lumi navigation menu, click Integrations > S3 ingest actions.

    Integration page

  2. In the Select or create an IAM key pane, click Select or create key > Create key.

  3. Enter the following information in the Create IAM key dialog:

    • Name: tutorial-s3
      Note that only the Name field is required to create the IAM key.
    • Description: IAM key for S3 tutorial
    • Environment: tutorial
    • Team: learning

    When you provide Environment and Team, Lumi assigns those values to the system attributes env and team, respectively. For more information, see System attributes.

    Create IAM key

  4. Click Create.

  5. Keep the Integrations page open. You'll need it to configure the S3 destination in the Create an S3 destination step.

    IAM Key S3 values

2. Create an S3 destination​

In this section, you create a remote S3 destination in Splunk Web to forward events to Lumi.

  1. In Splunk Web, go to Settings > Ingest actions > Destinations > New destination > S3.

  2. Enter the following details and leave all other fields unchanged:

    • S3 destination title: Name the destination as lumi-s3-tutorial.
    • S3 bucket name: Use the bucket name generated by Imply Lumi in the Create an IAM key section.
    • S3 endpoint: Use the endpoint provided by Lumi in the Create an IAM key section.

  3. Click Next.

  4. Configure authentication with the following details and leave all other fields unchanged:

    • Authentication method: Select Access key and secret key.
    • Access key ID: Enter the IAM key ID from Lumi that you created in the Create an IAM key step.
    • Secret access key: Enter the IAM key token from Lumi that you created in the Create an IAM key step.

  5. Click Test connection. A valid connection returns the message "Successfully connected to the server."

  6. Click Save to finish creating the destination.

  7. The newly created destination is now listed in your S3 destinations table.

3. Create an S3 ruleset​

In this section, you create a ruleset in Splunk Web to route events to Lumi. The ruleset uses the S3 destination you created earlier.

  1. In Splunk Web, go to Settings > Ingest actions > Rulesets > New ruleset.

  2. Enter the following details:

    • Name: lumi-s3-tutorial
    • Description: Ruleset to send events to Lumi

  3. Under Event stream, set the sourcetype to access_combined. This is the source type for Apache logs you will upload in the next section. Leave all other fields unchanged.

  4. Click Add rule > Route to destination.

  5. In the Immediately send to field, select the name of the S3 destination you created in the Create an S3 destination step.

  6. Click Apply and then Save.

  7. The newly created ruleset is now listed in your S3 ruleset table.

4. Send the data​

In this section, you upload a sample log file to Splunk to test event routing to Lumi.

  1. Download and save the example data file site_visitors.log.

  2. In Splunk Web, go to Settings > Add Data > Upload.

  3. Click Select File, then choose the site_visitors.log file you downloaded.

  4. Click Next.

  5. On the source type screen, set Source type to access_combined. This matches the source type defined in your ingest action ruleset.

  6. Leave all other settings unchanged and click Next.

  7. On the Review screen confirm the File name is site_visitors.log and Source type is access_combined.

  8. Click Submit.

  9. Successful upload returns the message: “File has been uploaded successfully.”

5. Preview data​

In this section, you preview the data you sent to Lumi and view the events in the explore view.

  1. Return to the S3 ingest actions integration page in Lumi. In the Preview incoming data section, view the newly added events.

    Preview incoming data

  2. Select Explore events. Lumi takes you to the explore view and applies search filters for your IAM key and the integration. You may need to adjust the time range selector and refresh the page to see the events.

    Explore events

  3. Select an event to view its attributes. For details on the attributes, see Event model.

    Event details pane

For information on searching events and filtering on event attributes, see Search events with Lumi.

Learn more​

For more information, see the following topics: