Skip to main content

2.17 Elasticsearch Sink Connector

Create a connector where Type is sink and Database is Elasticsearch

Pre-condition: CDC service status is Healthy

Steps to create a connector:​

Step 1: In the menu bar, select Data Platform > select Workspace Management > select Workspace name

Step 2: In the My services section, select CDC service

Step 3: On the CDC service detail screen > select the Connectors tab > click Create a connector

image-1

Step 4: Enter the information on the Connector Information screen:

  • Name (required): connector name

Note: The connector name can contain lowercase letters a-z or digits 0-9. In particular, spaces are not allowed — you can replace spaces with "-".

  • Type (required): Select sink
  • Database (required): Select Elasticsearch

image-2

Step 5: Click Next in the top-right corner of the screen to proceed to the Properties screen

  • Database Information - Enter the Database information
    • URL: Access address
    • Username: Account name
    • Password: Password

image-3

Click Test Connection to verify the connection from the Workspace to the entered Elasticsearch

  • Converter
    • Converter key: Select the key value for the converter
    • Converter key schema enable: Select whether or not to use a schema in the Converter key
    • Converter value: Select the value for the converter
    • Converter value schema enable: Select whether or not to use a schema in the Converter value

image-4

  • Kafka topic
    • Click the '+' to retrieve topic information
    • Note: limited to a maximum of 100 topics

image-5

Step 6. Click Next to proceed to the Additional Properties screen

  • Task:
    • Number of tasks: The maximum number of tasks that can run in parallel

image-6

  • Map table: Links the topic to a data table in the target database
    • Create new: Choose to create a new table, or select from the list of existing tables in the target database
    • Index: Select an index if you are not creating a new index, or enter an index name if you choose to create a new one

image-7

  • Error Tolerance: - Mode:
    • None: The Connector stops processing if an error occurs
    • All: Failed messages will be sent to the specified topic

image-8

Step 7. Click Next in the top-right corner of the screen to proceed to the Review screen

Step 8. Review the information, then click Create to finish creating the connector