I’m using Debezium MySQL Connector deployed in Kafka Connect to Stream MySQL Changes to Kafka Topics and from where we fetch these messages enrich data and push the data to another MySQL.
Both source and sink are MySQL.
I have couple of Columns in my source table with column datatype as TIMESTAMP.
Ex : create_time : 2021-10-06 09:32:46
I could see the above data in Kafka Message as below
Facing Issue with T and Z in the above String when the message is loaded into target database (MySQL),where the data type is TIMESTAMP as well.
I need create_time in kafka message as “create_time”:”2021-10-06 09:32:46″ without TZ
Error log :
- Row import failed with error: (“Incorrect datetime value: ‘2021-10-06T09:32:46Z’ for column ‘CREATE_TIME’ at row 1”, 1292)
I Understand that Debezium is converting Timestamp column data to ZonedDateTime which might be adding the TimeZone Information. (debezium.io/documentation/reference/1.7/connectors/mysql.html#mysql-temporal-types)
I tried org.apache.kafka.connect.transforms.TimestampConverter in Debezium Connector Configurations but this is not helping.
Currently I’m using org.apache.kafka.connect.json.JsonConverter in Kafka Connect Configurations.
Things to Note : I’m batching multiple messages received from Kafka, Construct a csv file and by using LOAD DATA I’m pushing the csv contents to Target database.
Read more here: Source link