How to convert a Splunk dashboard for federated search

This tutorial builds on How to search events with Splunk® to teach you how to:

• Create a dashboard in Splunk using Splunk queries.
• Clone the dashboard and update the clone to use federated search queries against Imply Lumi.

The steps assume that you've already:

• Configured a federated provider and index in Splunk to Lumi.
• Performed federated queries on Lumi events using the index "federated:search-tutorial".

To complete the steps, you use sample web traffic data from a fictional online store. For details about the dataset and its format, see the tutorial data overview.

The following diagram summarizes the end-to-end process of creating and updating a Splunk dashboard. Click any box in the diagram to jump to that step.

Prerequisites

Before you begin, download an up-to-date version of the sample log file and upload it to Lumi. This step is essential to ensure that you retrieve identical results when comparing your Splunk dashboard to your federated search dashboard.

To complete the tutorial, you need the following:

• Access to Lumi with the Viewer role or higher.
  For information on roles and permissions, see Manage roles.
• A Lumi IAM key.
  See Create an IAM key for details.
• A Splunk user with the admin_all_objects and indexes_edit capabilities. The following roles have these capabilities by default. See the Splunk documentation on security for federated search for more information.
  • Splunk Cloud: sc_admin
  • Splunk Enterprise: admin

1. Upload sample file to Splunk

In this step, you upload the sample log file to Splunk.

1. Download an up-to-date version of the sample log file.

2. In Splunk Web, go to Settings > Add Data.

3. Upload the sample log file using the following details:

   • Source type: csv.
   • Host field value: web-01.
   • Index: Default.

   Leave all other options unchanged.

4. Once the file is successfully uploaded, open the Search & Reporting app.

5. Run the following query to confirm that you can retrieve events from the sample data:

   earliest=-7d@d host=web-01 

   

2. Create Splunk dashboard

In this step, you create a Splunk dashboard showing site activity from the sample logs.

To create the dashboard:

1. In Splunk, perform the following query:

   earliest=-7d@d host=web-01
   | timechart span=12h count as requests

2. Click Visualization and select Column Chart.

3. Click Save As and select New dashboard.

4. Enter the following dashboard details:

   • Dashboard title: Site activity.
   • Dashboard type: Dashboard Studio with Absolute layout.
   • Panel title: Total requests per 12 hrs.

5. Click Save to Dashboard then View dashboard to see the dashboard and its first panel:

   

3. Add dashboard panels

To add panels to the dashboard:

1. Return to the search app in Splunk Web.

2. Run this query to show the top 10 users:

   earliest=-7d@d host=web-01
   | where user != "-"
   | top user
   1. Select the Pie Chart visualization.
   2. Click Save As > Existing dashboard > Site activity.
   3. Add panel title Top 10 users.
   4. Click Save to dashboard.
   5. Close the View dashboard dialog.

3. Run this query to show a breakdown of HTTP statuses:

   earliest=-7d@d host=web-01
   | eval status_group=if(status>=500, "5xx Server Error",
      if(status>=400, "4xx Client Error",
      if(status>=300, "3xx Redirect",
      if(status>=200, "2xx Success", "Other"))))
   | stats count by status_group
   1. Click Save As > Existing dashboard > Site activity.
   2. Add panel title HTTP status breakdown.
   3. Click Save to dashboard.
   4. Close the View dashboard dialog.

4. Run this query to show the 10 most viewed pages:

   earliest=-7d@d host=web-01
   | stats count by uri_path
   | sort -count
   | head 10
   1. Click Save As > Existing dashboard > Site activity.
   2. Add panel title Most viewed pages.
   3. Click Save to dashboard.
   4. Close the View dashboard dialog.

5. Run this query to display a running sum of bytes transferred:

   earliest=-7d@d host=web-01
   | sort _time
   | streamstats sum(bytes) as running_total_bytes
   | table _time, bytes, running_total_bytes
   1. Select the Line Chart visualization.
   2. Set Format > Multi-series Mode > Yes.
   3. Click Save As > Existing dashboard > Site activity.
   4. Add panel title Running sum of bytes transferred.
   5. Click Save to dashboard.

6. Click View dashboard:

   

The default dashboard layout shows panels stacked vertically. You'll copy some example formatting in the next step.

4. Clone and convert dashboard

In this step, you clone the Site Activity dashboard and update the clone to use federated queries against Lumi.

To clone the dashboard and update the clone:

1. In Splunk Web, go to Dashboards.

2. Click the Actions drop-down arrow next to the Site Activity dashboard and select Clone.

3. Enter the title Site Activity - Federated and click Clone Dashboard.

4. When you see Dashboard has been cloned, click View.

5. Click Edit to edit the dashboard.

6. Click the source icon in the editor toolbar:

   

7. In the source, prepend query definitions in the dataSources object with the federated index to route them through the Lumi federated provider you set up in How to search events with Splunk. For example, this tutorial uses search-tutorial:

   "query": "index=federated:search-tutorial earliest=-7d@d host=web-01 | stats count by uri_path | sort -count | head 10"

   For this tutorial you can replace everything in the source window with the code below, which prepends queries with index=federated:search-tutorial and applies dashboard formatting.

   Federated dashboard source

   {
    "title": "Site Activity - Federated",
    "description": "Site activity for the past 7 days.",
    "inputs": {
        "input_global_trp": {
            "options": {
                "defaultValue": "-24h@h,now",
                "token": "global_time"
            },
            "title": "Global Time Range",
            "type": "input.timerange"
        }
    },
    "defaults": {
        "dataSources": {
            "ds.search": {
                "options": {
                    "queryParameters": {
                        "earliest": "$global_time.earliest$",
                        "latest": "$global_time.latest$"
                    }
                }
            }
        },
        "visualizations": {
            "global": {
                "showProgressBar": true
            }
        }
    },
    "visualizations": {
        "viz_6NuRD8bu": {
            "dataSources": {
                "primary": "ds_0ttsx0Sr"
            },
            "options": {
                "count": 20,
                "dataOverlayMode": "none",
                "drilldown": "none",
                "showInternalFields": false,
                "showRowNumbers": false
            },
            "title": "Most viewed pages",
            "type": "splunk.table"
        },
        "viz_HCMCFpkL": {
            "dataSources": {
                "primary": "ds_lBUW8SqQ"
            },
            "options": {
                "collapseThreshold": 0.01,
                "labelDisplay": "valuesAndPercentage"
            },
            "title": "Top 10 users",
            "type": "splunk.pie"
        },
        "viz_TobsS7qs": {
            "containerOptions": {
                "visibility": {}
            },
            "dataSources": {
                "primary": "ds_l3rorak9"
            },
            "options": {
                "dataValuesDisplay": "all",
                "legendDisplay": "right",
                "legendTruncation": "ellipsisMiddle",
                "seriesColors": [
                    "#009CEB",
                    "#7B56DB",
                    "#00CDAF",
                    "#DD9900",
                    "#FF677B",
                    "#CB2196",
                    "#813193",
                    "#0051B5",
                    "#008C80",
                    "#99B100",
                    "#FFA476",
                    "#FF6ACE",
                    "#AE8CFF",
                    "#00689D",
                    "#00490A",
                    "#465D00",
                    "#9D6300",
                    "#F6540B",
                    "#FF969E",
                    "#E47BFE"
                ],
                "showIndependentYRanges": false,
                "showOverlayY2Axis": false,
                "showRoundedY2AxisLabels": false,
                "showSplitSeries": false,
                "showY2MajorGridLines": true,
                "stackMode": "auto",
                "xAxisLabelRotation": 0,
                "y2AxisAbbreviation": "auto",
                "y2AxisScale": "linear",
                "y2AxisTitleVisibility": "show",
                "yAxisAbbreviation": "auto",
                "yAxisScale": "linear"
            },
            "title": "Total requests per 12 hrs",
            "type": "splunk.column"
        },
        "viz_XwHID0vF": {
            "dataSources": {
                "primary": "ds_DkqxFW87"
            },
            "options": {
                "count": 20,
                "dataOverlayMode": "none",
                "drilldown": "none",
                "showInternalFields": false,
                "showRowNumbers": false
            },
            "title": "HTTP status breakdown",
            "type": "splunk.table"
        },
        "viz_b2PKCAfj": {
            "dataSources": {
                "primary": "ds_rmGriIam"
            },
            "options": {
                "dataValuesDisplay": "off",
                "legendDisplay": "right",
                "legendMode": "standard",
                "legendTruncation": "ellipsisMiddle",
                "lineWidth": 2,
                "nullValueDisplay": "gaps",
                "showIndependentYRanges": false,
                "showOverlayY2Axis": false,
                "showRoundedY2AxisLabels": false,
                "showSplitSeries": true,
                "showY2MajorGridLines": true,
                "xAxisLabelRotation": 0,
                "xAxisTitleVisibility": "show",
                "y2AxisAbbreviation": "auto",
                "y2AxisScale": "linear",
                "y2AxisTitleVisibility": "show",
                "yAxisAbbreviation": "auto",
                "yAxisScale": "linear",
                "yAxisTitleVisibility": "show"
            },
            "title": "Running sum of byes transferred",
            "type": "splunk.line"
        }
    },
    "dataSources": {
        "ds_0ttsx0Sr": {
            "name": "Most viewed pages search",
            "options": {
                "query": "index=federated:search-tutorial earliest=-7d@d host=web-01 | stats count by uri_path | sort -count | head 10",
                "queryParameters": {
                    "earliest": "-30m@m",
                    "latest": "now",
                    "sampleRatio": 1
                }
            },
            "type": "ds.search"
        },
        "ds_50bkSAJ9": {
            "name": "HTTP status breakdown search",
            "options": {
                "query": "index=federated:search-tutorial earliest=-7d@d host=web-01 | where user != \"-\" | top user",
                "queryParameters": {
                    "earliest": "-30m@m",
                    "latest": "now",
                    "sampleRatio": 1
                }
            },
            "type": "ds.search"
        },
        "ds_DkqxFW87": {
            "name": "HTTP status breakdown search",
            "options": {
                "query": "index=federated:search-tutorial earliest=-7d@d host=web-01 | eval status_group=if(status>=500, \"5xx Server Error\", if(status>=400, \"4xx Client Error\", if(status>=300, \"3xx Redirect\", if(status>=200, \"2xx Success\", \"Other\")))) | stats count by status_group",
                "queryParameters": {
                    "earliest": "-30m@m",
                    "latest": "now",
                    "sampleRatio": 1
                }
            },
            "type": "ds.search"
        },
        "ds_l3rorak9": {
            "name": "Total requests per 12 hrs - Column chart search",
            "options": {
                "query": "index=federated:search-tutorial earliest=-7d@d host=web-01 | timechart span=12h count as requests",
                "queryParameters": {
                    "earliest": "0",
                    "sampleRatio": 1
                }
            },
            "type": "ds.search"
        },
        "ds_lBUW8SqQ": {
            "name": "Top 10 users search",
            "options": {
                "query": "index=federated:search-tutorial earliest=-7d@d host=web-01 | where user != \"-\" | top user",
                "queryParameters": {
                    "earliest": "-30m@m",
                    "latest": "now",
                    "sampleRatio": 1
                }
            },
            "type": "ds.search"
        },
        "ds_rmGriIam": {
            "name": "Running sum of byes transferred search",
            "options": {
                "query": "index=federated:search-tutorial earliest=-7d@d host=web-01 | sort _time | streamstats sum(bytes) as running_total_bytes | table _time, bytes, running_total_bytes",
                "queryParameters": {
                    "earliest": "-30m@m",
                    "latest": "now",
                    "sampleRatio": 1
                }
            },
            "type": "ds.search"
        }
    },
    "layout": {
        "globalInputs": [
            "input_global_trp"
        ],
        "layoutDefinitions": {
            "layout_1": {
                "options": {
                    "display": "auto",
                    "height": 1480,
                    "width": 1440
                },
                "structure": [
                    {
                        "item": "viz_TobsS7qs",
                        "position": {
                            "h": 390,
                            "w": 810,
                            "x": 10,
                            "y": 10
                        },
                        "type": "block"
                    },
                    {
                        "item": "viz_XwHID0vF",
                        "position": {
                            "h": 300,
                            "w": 300,
                            "x": 440,
                            "y": 410
                        },
                        "type": "block"
                    },
                    {
                        "item": "viz_6NuRD8bu",
                        "position": {
                            "h": 300,
                            "w": 420,
                            "x": 10,
                            "y": 410
                        },
                        "type": "block"
                    },
                    {
                        "item": "viz_b2PKCAfj",
                        "position": {
                            "h": 300,
                            "w": 610,
                            "x": 750,
                            "y": 410
                        },
                        "type": "block"
                    },
                    {
                        "item": "viz_HCMCFpkL",
                        "position": {
                            "h": 390,
                            "w": 530,
                            "x": 830,
                            "y": 10
                        },
                        "type": "block"
                    }
                ],
                "type": "absolute"
            }
        },
        "options": {},
        "tabs": {
            "items": [
                {
                    "label": "New tab",
                    "layoutId": "layout_1"
                }
            ]
        }
    },
    "applicationProperties": {
        "collapseNavigation": false,
        "hideEdit": false,
        "hideExport": false
    }
   }

8. Click Apply and close.

The dashboard refreshes to show federated query results from Lumi:

5. Compare dashboards

You can now compare the Site Activity dashboard (Splunk queries) against the Site Activity - Federated dashboard (Lumi). The results should be identical. If they're not, make sure you've uploaded the same sample data to both Lumi and Splunk.

Note that if you update your own Splunk dashboard for federated search, it may not appear exactly the same if the commands and functions in your queries aren't yet fully supported for use with Lumi. See Splunk federated search reference for an up-to-date list of supported commands.

Learn more

See the following topics for more information:

• Set up Splunk federated search for configuring federated search in Splunk.
• Splunk federated search syntax for operators and syntax supported by Lumi in federated search.
• Splunk dashboard studio for creating and customizing Splunk studio dashboards.