Skip to main content

Send events with OTLP

OpenTelemetry (OTel) is an open-source observability framework you can use to collect and export your telemetry data. You can send events to Imply Lumi using the OpenTelemetry Protocol (OTLP).

This topic provides details on configuring event forwarding using OTLP.

OTLP integration​

OTLP describes the data transfer protocol, including encoding, transport, and delivery mechanism of telemetry data. The mechanism by which you send OTLP data depends on your specific system setup and requirements. For example, you can send OTLP data directly from a service or through the OTel collector. The OTel collector is a vendor-neutral application to send events to observability backends. It's a standalone service that acts as an intermediary between the telemetry source and an observability backend.

For development or small-scale environments, you may choose to instrument a service to send data directly to Lumi using OTLP. For broader scenarios, including production environments, the OpenTelemetry community recommends using the OTel collector. The collector offloads data from the service and handles retries, batching, encrypting, and data filtering. You can also use the OTel collector with the HEC integration.

The OTLP integration in Lumi uses the following specifications:

  • Encoding: JSON only. The binary encoding is not supported.
  • Transport protocol: HTTP only. The gRPC framework is not supported.

The following diagram shows a high-level overview of the OTLP integration:

Lumi OTLP

For other approaches to send events, see Send events to Lumi.

Prerequisites​

To send events to Lumi using OTLP, you need the following:

  • Access to Lumi with the Data manager role or higher. For information on roles and permissions, see Manage roles.

  • Lumi IAM key. See Create an IAM key for details.

    The IAM key authenticates connections to send events to Lumi. It also enriches incoming events with the system attributes for environment and team.

  • Port 443 open for outbound TCP traffic on the machine forwarding the events.

Configure event forwarding​

This section describes how to forward events to Lumi using OTLP. You can send events using any application that supports sending HTTP requests. The examples demonstrate how to use the OTel collector, which is suitable for production-level scenarios, and the command-line tool curl to quickly test a connection to Lumi.

To configure event forwarding, complete the following steps:

  1. From the Lumi navigation menu, click Integrations > OpenTelemetry Protocol (OTLP).
  2. Select or create an IAM key.
  3. For the application sending the requests, add details from the Lumi integration.

The following sections provide detailed instructions for example applications. To use the examples, replace the variables with your own values:

  • IAM_KEY_TOKEN: IAM key token provided by Lumi.
    For example, 229a2561-0000-0000-0000-bc433de16f89
  • LUMI_ENDPOINT: Lumi endpoint.
    For example, https://otel.api.lumi.imply.io/v1/logs
info

To send an event over HTTP using the OTLP integration, append the path /v1/logs to the URL. For example, https://otel.us1.api.lumi.imply.io/v1/logs. Note that when you configure an otlphttp exporter using the OTel collector, you don't include /v1/logs since the exporter automatically appends that path to the endpoint.

Example with OTel collector​

The following example shows how to configure the OTel collector to send data to Lumi using OTLP.

For an end-to-end tutorial using the OTel collector, see Send events with the OTel collector. While the tutorial uses the HEC integration, you can adapt it to use the OTLP integration by applying the following configuration.

The OTel collector configuration involves creating an exporter and enabling it in the pipelines service.

  1. In the configuration for your OTel collector, add a section in exporters with exporter type otlphttp:

    exporters:
      <any existing exporters>
        ...
      otlphttp/logs_lumi:
        endpoint: "LUMI_ENDPOINT"
        encoding: json
        headers:
          authorization: "Bearer IAM_KEY_TOKEN"

    The name of an OTel collector exporter follows the format TYPE/NAME, where the name uniquely identifies the component. The preceding description uses the logs_lumi name, but you can customize it to your preference.

  2. Enable the exporter by listing it in service.pipelines.logs.exporters:

    service:
      ...
      pipelines:
        logs:
          receivers: ...
          processors: ...
          exporters: [..., otlphttp/logs_lumi]

The following example shows the relevant stanzas of an OTel collector configuration:

...

exporters:
  <any existing exporters>
    ...
  otlphttp/logs_lumi:
    endpoint: "https://otel.api.lumi.imply.io"
    encoding: json
    headers:
      authorization: "Bearer 229a2561-0000-0000-0000-bc433de16f89"

...

service:
  ...
  pipelines:
    logs:
      receivers: ...
      processors: ...
      exporters: [..., otlphttp/logs_lumi]

Example with curl​

You can use curl to send a test event to Lumi. This command-line tool is suitable for quick tests and simple API calls.

In the request, send the data as an OTLP log in JSON format. Lumi looks for the event message in resourceLogs.scopeLogs.logRecords.body.stringValue. For more information, see the OTel docs for OTLP JSON encoding and an example request payload.

The following code block shows an example HTTP POST request for curl. It sends the event called Simple event message from curl without any user attributes. To send an event from the present time, obtain the present time in units of nanoseconds from Unix time, and update the value in timeUnixNano.

curl -X POST \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer IAM_KEY_TOKEN" \
     -d '{
       "resourceLogs": [{
         "scopeLogs": [{
           "logRecords": [{
             "timeUnixNano": "1752700849000000000",
             "body": { "stringValue": "Simple event message from curl" }
           }]
         }]
       }]
     }' \
  LUMI_ENDPOINT/v1/logs

To send user attributes with the event, include them in one of the following fields. Lumi assigns them all as user attributes on the event.

  • resourceLogs.resource.attributes
  • resourceLogs.scopeLogs.scope.attributes
  • resourceLogs.scopeLogs.logRecords.attributes
View example request with user attributes

The following curl command sends the event for the JSON payload in the OTel example.

curl -X POST \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer 229a2561-0000-0000-0000-bc433de16f89" \
     -d '{
  "resourceLogs": [
    {
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": {
              "stringValue": "my.service"
            }
          }
        ]
      },
      "scopeLogs": [
        {
          "scope": {
            "name": "my.library",
            "version": "1.0.0",
            "attributes": [
              {
                "key": "my.scope.attribute",
                "value": {
                  "stringValue": "some scope attribute"
                }
              }
            ]
          },
          "logRecords": [
            {
              "timeUnixNano": "1752699720761000000",
              "observedTimeUnixNano": "1544712660300000000",
              "severityNumber": 10,
              "severityText": "Information",
              "traceId": "5B8EFFF798038103D269B633813FC60C",
              "spanId": "EEE19B7EC3C1B174",
              "body": {
                "stringValue": "Example log record"
              },
              "attributes": [
                {
                  "key": "string.attribute",
                  "value": {
                    "stringValue": "some string"
                  }
                },
                {
                  "key": "boolean.attribute",
                  "value": {
                    "boolValue": true
                  }
                },
                {
                  "key": "int.attribute",
                  "value": {
                    "intValue": "10"
                  }
                },
                {
                  "key": "double.attribute",
                  "value": {
                    "doubleValue": 637.704
                  }
                },
                {
                  "key": "array.attribute",
                  "value": {
                    "arrayValue": {
                      "values": [
                        {
                          "stringValue": "many"
                        },
                        {
                          "stringValue": "values"
                        }
                      ]
                    }
                  }
                },
                {
                  "key": "map.attribute",
                  "value": {
                    "kvlistValue": {
                      "values": [
                        {
                          "key": "some.map.key",
                          "value": {
                            "stringValue": "some value"
                          }
                        }
                      ]
                    }
                  }
                }
              ]
            }
          ]
        }
      ]
    }
  ]
     }' \
  https://otel.us1.api.lumi.imply.io/v1/logs

Note the user attributes stored on the event in the following screenshot:

Event ingested with attributes

Check Lumi for events​

Once you configure event forwarding and send events, you can preview the incoming data in Lumi:

  1. From the Lumi navigation menu, click Integrations > OpenTelemetry Protocol.

  2. In Select or create an IAM key, select your key.

  3. In Preview incoming data, view the events coming in to Lumi. Lumi automatically refreshes the preview pane to display the latest events.

  4. Click Explore events to see more events associated with the IAM key. The explore view populates the search bar with your IAM key ID and the receiver type. For example:

    #iamKeyId=229a2561-0000-0000-0000-bc433de16f89 #receiver=otel

    Adjust the time filter to choose the range of data displayed.

Once events start flowing into Lumi, you can search them. See Search events with Lumi for details and information on supported search syntax.

Learn more​

See the following topics for more information: