Skip to main content

Connect to Confluent Cloud

To ingest data from Confluent Cloud into Imply Polaris, create a Confluent Cloud connection and use it as the source of an ingestion job.

The following characteristics apply to ingestion from Confluent Cloud to Polaris:

  • Exactly-once guarantees.
  • Occurs across the Internet with TLS encryption and SASL authentication.

This topic provides reference information to create a Confluent Cloud connection.

tip

For an end-to-end guide to Confluent Cloud ingestion in Polaris, see Guide for Confluent Cloud ingestion.

Create a connection​

Create a Confluent Cloud connection as follows:

  1. Click Sources from the left navigation menu.
  2. Click Create source and select Confluent Cloud.
  3. Enter the connection information.
  4. Click Test connection to confirm that the connection is successful.
  5. Click Create connection to create the connection.

The following screenshot shows an example connection created in the UI. For more information, see Create a connection.

Polaris Confluent Cloud connection

Connection information​

Follow the steps in Create a connection to create the connection. The connection requires the following information from Confluent Cloud:

  • Topic name: The topic name or a regular expression that identifies the Kafka topics.
    For details on creating a single connection that ingests from multiple Kafka topics, see Multiple topics per connection.

  • Bootstrap servers: A list of one or more host and port pairs representing the addresses of brokers in the Kafka cluster. This list should be in the form host1:port1,host2:port2,... For details on where to find the bootstrap server in Confluent Cloud, see Access cluster settings in the Confluent Cloud Console.

    You can also use the Confluent Cloud API to read a cluster and find its bootstrap server information in the kafka_bootstrap_endpoint of the cluster object.

Authentication​

For Polaris to access your data from Confluent Cloud, create a resource API key in Confluent Cloud, and supply the API key and secret in the Polaris connection.

Ensure that the API key has access to the Kafka cluster. Note that resource-specific API keys in Confluent Cloud require an access control list that restricts access.

For information on creating and managing API keys in Confluent Cloud, see Use API Keys to Control Access.

Learn more​

See the following topics for more information: