I need to load multiple Kafka topics into AWS S3. For example, I will get the Kafka topics in every 30sec aa Avro data format. I wanted to store those topics into AWS S3 under current_date folder and I need to append the data. Can any one please let me know on how to do that?
This process will happen every day.
I know that Confluent Kafka connector is used to connect Kafka and S3.
It would be very helpful if any of you point to some use case link, docs or videos or explain here.
Thanks in advance.
Read more here: Source link