apache kafka – Migrate from Mirrormaker 1.0 to 2.0 with preserving consumer offset from MM1

MirrorMaker2 doesn’t use consumer groups since it’s a Source Connector

You’d therefore need to find some other way to consume the __consumer_offsets topic or parse output from kafka-consumer-groups command, filtering for your MM1 groups, and then create events that MM2 uses to track its progress in the “internal” Connect mm2-offsets topic

Blog here about how you’d do this with a different source connector rmoff.net/2019/08/15/reset-kafka-connect-source-connector-offsets/

Keep in mind that MM2 also translates the consumer offsets topic and deals with heartbeats/timestamps, which are not part of MM1, making it difficult to do a complete migration

Read more here: Source link