• Developer guide
  • API reference

›Ingestion sources

Getting started

  • Introduction to Imply Polaris
  • Quickstart
  • Navigate the console
  • Key concepts

Ingestion sources

  • Ingestion sources overview
  • Supported data formats
  • Create a connection
  • Ingest from files
  • Ingest from S3
  • Ingest from Kinesis
  • Ingest from Confluent Cloud
  • Kafka Connector for Imply Polaris
  • Push event data

Tables and data

  • Overview
  • Introduction to tables
  • Table schema
  • Create an ingestion job
  • Timestamp expressions
  • Data partitioning
  • Introduction to rollup
  • Approximation algorithms
  • Replace data

Analytics

  • Overview
  • Manage data cubes
  • Visualize data
  • Data cube dimensions
  • Data cube measures
  • Dashboards
  • Create a dashboard
  • Visualizations reference
  • Set up alerts
  • Set up reports
  • Embed visualizations
  • Query data

Monitoring

  • Overview

Management

  • Overview
  • Pause and resume a project

Billing

  • Overview
  • Polaris plans
  • Estimate project costs

Usage

  • Overview

Security

    Polaris access

    • Overview
    • Invite users to your organization
    • Permissions reference
    • Manage user groups
    • Enable SSO
    • SSO settings reference
    • Map IdP groups

    Secure networking

    • Connect to AWS

Developer guide

  • Overview
  • Authentication

    • Overview
    • Authenticate with API keys
    • Authenticate with OAuth
  • Manage users and groups
  • Migrate deprecated resources
  • Create a table
  • Define a schema
  • Upload files
  • Create an ingestion job
  • Ingestion sources

    • Ingest from files
    • Ingest from a table
    • Get ARN for AWS access
    • Ingest from Amazon S3
    • Ingest from Amazon Kinesis
    • Ingest from Confluent Cloud
    • Push event data
    • Kafka Connector for Imply Polaris
    • Kafka Connector reference
  • Filter data to ingest
  • Ingest nested data
  • Ingest and query sketches
  • Query data
  • Update a project
  • Link to BI tools
  • Connect over JDBC
  • Query parameters reference
  • API documentation

    • OpenAPI reference
    • Query API

Product info

  • Release notes
  • Known limitations
  • Druid extensions

Ingest from Confluent Cloud

You can create a connection to Confluent Cloud, a fully managed, cloud-native service for Apache Kafka to ingest data into Imply Polaris.

The following characteristics apply to ingestion from Confluent Cloud to Polaris:

  • Exactly-once guarantees.
  • Occurs across the Internet with TLS encryption and SASL authentication.

Confluent Cloud connection information

  • Topic name: The name of the Confluent Cloud topic that contains the event data.

  • Bootstrap servers: A list of one or more host and port pairs representing the addresses of brokers in the Kafka cluster. This list should be in the form host1:port1,host2:port2,... For details on where to find the bootstrap server in Confluent Cloud, see Access cluster settings in the Confluent Cloud Console.

    You can also use the Confluent Cloud API to read a cluster and find its bootstrap server information in the kafka_bootstrap_endpoint of the cluster object.

  • A Confluent Cloud API key with access to Kafka resources: The API key consists of a key and a secret. The Polaris connection takes both the key and the secret to access data through the connection. Note that resource-specific API keys in Confluent Cloud require an access control list that restricts access. For information on creating and managing API keys in Confluent Cloud, see Use API Keys to Control Access.

The following screenshot shows an example of a successful connection created in the UI:

Polaris sources

When you have your table, connection, and ingestion job set up for Confluent Cloud, Polaris automatically ingests data from Kafka topic as data enters the topic defined in the connection.

Learn more

To learn how to ingest data from Confluent Cloud using the Polaris API, see Ingest data from Confluent Cloud by API.

← Ingest from KinesisKafka Connector for Imply Polaris →
  • Confluent Cloud connection information
  • Learn more
Key links
Try ImplyApache Druid siteImply GitHub
Get help
Stack OverflowSupportContact us
Learn more
BlogApache Druid docs
Copyright © 2023 Imply Data, Inc