Integrate with Elastic stack

This topic provides a sample configuration for reporting Polaris performance metrics to Elasticsearch.

The topic assumes that you are using Elasticsearch for storing and searching your data, Kibana for visualizing and managing the data, and Metricbeat for collecting metrics.

Prerequisites

• An Elasticsearch instance. Refer to the Elasticsearch documentation for installation instructions.
• A Kibana instance configured to run against the same major version release as Elasticsearch. Refer to the Kibana documentation for installation instructions.
• A Metricbeat instance with Openmetrics module enabled. Refer to the Metricbeat documentation for installation instructions.
• A Polaris API key with the AccessMetrics permission. See Authenticate with API keys to obtain an API key and assign service account permissions. For more information on permissions, visit Permissions reference.

Instructions

The following is an example configuration for the Metricbeat Openmetrics module to collect metrics about your Polaris environment. For more advanced use cases, refer to the Openmetrics module documentation.

In the Metricbeat directory named modules.d, edit the openmetrics.yml file to connect to the Polaris Metrics export API endpoint:

# Module: openmetrics

- module: openmetrics
  metricsets: ['collector']
  # Polaris updates metrics every minute, so this period is set to 60 seconds.
  period: 60s
  # Replace ORGANIZATION_NAME with the name of your organization.
  hosts: ["https://ORGANIZATION_NAME.api.imply.io"]

  # The name of the service the data is collected from. 
  # This can be changed.
  service.name: imply

  metrics_path: /v1/metrics/export
  username: apikey
  # Replace POLARIS_API_KEY with your API key. 
  # Ensure that the API key has the AccessMetrics permission.
  password: POLARIS_API_KEY

  metrics_filters:
    include: []
    exclude: []

Test your integration in Docker

To test this integration locally, you can use a single-node Elasticsearch cluster in a Docker container.

1. Follow the steps in Run Kibana on Docker for development to start Kibana and connect it to your Elasticsearch container.

2. Access Kibana through the web application on port 5601.

3. On the home page, click Add integrations.

4. Select OpenMetrics Metrics integration.

5. Follow the steps displayed in the UI to install Metricbeat.

6. Modify metricbeat.yml to set the connection information:

   output.elasticsearch:
   
     # Array of hosts to connect to. 
     # Replace ES_URL with your Elasticsearch host. For example: "localhost:9200".
     hosts: ["ES_URL"]
   
     # Protocol - either `http` (default) or `https`.
     # Replace ES_PASSWORD with your elastic user password.
     protocol: "https"
     username: "elastic"
     password: "ES_PASSWORD"
   
     # If using Elasticsearch's default certificate.
     # Replace ES_CERT_FINGERPRINT with your HTTP CA certificate SHA-256 fingerprint.
     ssl.ca_trusted_fingerprint: "ES_CERT_FINGERPRINT"
      
   setup.kibana:
     # Replace KIBANA_URL with your Kibana host. For example: "localhost:5601". 
     host: "KIBANA_URL"

7. Follow the steps displayed in the UI to enable and configure the Openmetrics module.

8. Modify the settings in the modules.d/openmetrics.yml file to connect to the Polaris Metrics export API endpoint.

9. Start Metricbeat:

   ./metricbeat setup
   ./metricbeat -e

10. In the UI, go to Dashboard > Create dashboard. Click Create visualization. Follow Kibana’s Dashboard and visualizations documentation to visualize your Polaris performance metrics. Refer to the Metrics reference documentation for a list of available metrics.

Learn more

See the following topics for more information:

• Monitor performance metrics
• Metrics reference