Skip to main content

2.7 Clickhouse (logs) sink connector

Use case: creating a connector where Type is sink and Database is ClickHouse

Pre-condition: CDC service status is healthy

Steps to create a connector:​

Step 1: In the menu bar, select Data Platform > select Workspace Management > select Workspace name

Step 2: In the My services section, select CDC service

  • Note: You can also go directly to the CDC service by: selecting Data Platform in the menu bar > selecting CDC service > clicking a CDC service name

Step 3: On the CDC service detail screen > select the Connectors tab > click Create a connector

image-1

Step 4: In the connector creation form, enter the information on the Connector Information screen:

  • Name (required): connector name

Note: The connector name can contain lowercase letters a-z or digits 0-9. In particular, spaces are not allowed — you can replace spaces with "-".

  • Type (required): select sink
  • Database (required): select ClickHouse(log)

image-2

Step 5: Click Next to proceed to the Properties screen

Enter the following information:

  • If you select Manual configuration - fill in the following information:
    • Host Name (required): Hostname or IP of ClickHouse
    • Port (required): ClickHouse server port, default: 8123.
    • Database name (required): The target database that the Connector will sink data into
    • Username (required): Username used by the Connector
    • Password (required): Password used by the Connector

image-3

  • If you select From FPT Database Engine - fill in the following information:
    • Database (required): Select the Database
    • Host Name (required): Hostname or IP of ClickHouse
    • Port (required): ClickHouse server port, default: 8123.
    • Database name (required): The target database that the Connector will sink data into
    • Username (required): Username used by the Connector
    • Password (required): Password used by the Connector

image-4

Click Test Connection to verify the connection from the Workspace to the entered Database

  • Converter
    • Converter key: select the key value for the converter
    • Converter key schema enable: select whether or not to use a schema in the Converter key
    • Converter value: select the value for the converter
    • Converter value schema enable: select whether or not to use a schema in the Converter value

image-5

  • Topics (required): The list of topics the Connector will consume and sink data into ClickHouse, separated by ","

image-6

Step 6: Click Next in the top-right corner of the screen to proceed to the Additional Properties screen

Fill in the following information:

  • Tasks max (required): Maximum number of tasks for the connection

image-7

  • Topic 1: The name of the topic the Connector will consume and sink data into ClickHouse

  • Table 1: The name of the table to listen for data changes from PostgreSQL

image-8

  • Mode (required): The Connector's behavior when a message cannot be processed
    • None: The Connector will skip messages that cannot be sinked into the database
    • All: Failed messages will be sent to the specified topic

image-9

Step 7. Click Next to proceed to the Review screen

image-10

Step 8: Review the information and click the Create button to finish creating the connector.