NIFI not streaming all messages between KAFKA topics

I’m using NIFI to stream messages between Kafka topics, using NIFI’s ConsumeKafka_2_ and PublishKafka_2_6 . The ConsumeKafka_2_6 defined with Offset Reset : earliest .

The source Topic x1 has 22 million messages but only 12 million messages streaming into the new destination Topic x2 .

Topic x1 and Topic x2 have the same definition.

Any idea why not all the 22 million messages streaming into the destination Topic x2 ?

One important thing,the cleanup.policy on topics is compact

Read more here: Source link