Skip to main content

Create a MongoDB sink connector

Use a MongoDB sink connector to keep collections in a MongoDB database up to date with the changes a source connector captures. The connector reads the Kafka topics you select and writes each topic to the collection you map it to.

Prerequisites​

  • A CDC Service whose status is Healthy. See Create a CDC Service.
  • Topics with change data, written by a source connector such as a MongoDB source connector.
  • The connection URL of a MongoDB deployment that the workspace can reach, and a user with the readWrite role on the target database.

Create the connector​

  1. Select Data platform → Workspace management, then select your workspace.

  2. On the Overview tab, under Services, select the CDC card.

  3. Select the Connectors tab, then select Create Connector.

    Connectors tab of a CDC Service in Healthy status, with no connector yet and the Create Connector button

  4. On the Connector Information screen, enter a Name, select sink in Type, and select MongoDB in Database.

    Connector Information screen of the Create Connector form with the Name, Type, and Database fields

  5. Select Next Step. On the Properties screen, enter the connection details, then select Test connection. Next Step becomes available after the test succeeds.

    Database Information section with Manual configuration selected and the Connection URL, Username, Password, and Database fields

    Database Information section with From FPT Database Engine selected and the Database Name list above the connection fields

  6. On the same screen, choose a Converter class and set the four converter fields. Under Kafka topic, select +, then select the topics to read in the Select topics panel.

    Converter section with the Converter class, Converter key, Converter value, and schema enable lists

    Kafka topic section with one topic in the list, the add and clear buttons, the Filter box, and the Topic and Action columns

  7. Select Next Step. On the Additional properties screen, set the fields in the Task, Database source, and Map table sections.

    Map table section with a topic mapped to users in the Index column, Create new turned on, and the Enable filter errors checkbox

  8. Select Next Step and check the settings on the Review screen.

    Review screen with the Basic Information, Database Information, Database source, Kafka topic, Task, and Converter sections and the Create button

  9. Select Create.

    The portal returns to the Essential Information tab of the CDC Service. On the Connectors tab, the new connector shows the status Running, with every Tasks counter at 0 until its task starts, which can take up to about 2.5 minutes.

Connector settings​

Connector information​

FieldRequiredDescription
NameYesThe connector name. Use lowercase letters, digits, and hyphens (-) only. Upper-case letters, _, ., and spaces are rejected with "Name has invalid characters".
TypeYesSelect sink.
DatabaseYesSelect MongoDB.

Properties​

In the Database Information section, choose From FPT Database Engine or Manual configuration. Manual configuration is selected by default. With From FPT Database Engine, select your instance in Database Name. The portal fills Connection URL, which you cannot edit, and you enter Username, Password, and Database. See Connection mode.

FieldRequiredDescription
Database NameYesShown with From FPT Database Engine only. The MongoDB instance in FPT Database Engine. Enter the database to write to in Database.
Connection URLYesThe MongoDB connection URI.
UsernameYesThe user that the connector connects as.
PasswordYesThe password of that user.
DatabaseYesThe MongoDB database that the connector writes to. Use letters, digits (0-9), underscores (_), and hyphens (-) only. Spaces are not allowed.
Converter classYesThe converter class for this connector. The portal offers MongoDB. The four converter fields stay greyed out until you choose a class. See Converter.
Converter key, Converter key schema enable, Converter value, Converter value schema enableYesHow the connector reads the keys and values of the messages, and whether they carry a schema. Converter key offers org.apache.kafka.connect.json.JsonConverter and org.apache.kafka.connect.storage.StringConverter, and Converter value only org.apache.kafka.connect.json.JsonConverter. Both schema enable fields offer true and false.
Kafka topicYesThe topics to read, up to 100. Select + to open the Select topics panel, select the topics, then select Select. See Topics.

Additional properties​

The MongoDB sink has no Timezone setting and no Error Tolerance section.

FieldRequiredDescription
Tasks maxYesHow many tasks the connector runs in parallel when the topics have more than one partition. With more than one task, records can arrive out of order. Default 1. See Tasks max.
Type (under Database source)YesThe type of the source database that the topics come from: None, PostgreSQL, MySQL, MariaDB, SQLServer, or MongoDB. The field has no default.
Topic, Create new, Index (under Map table)YesThe collection that receives the records of each topic. The form labels this column Index, but it takes a collection name, and the Review screen shows it as Collection. Create new is on by default, and Index is prefilled with the table name in the topic, which becomes the collection name. Keep it on to create the collection, or turn it off to select an existing one. If you turn it off and on again, type the name again. An empty Index gives "Table is required". Select Enable filter errors to list only the rows with errors. See Map table.

Next steps​