NIFI not streaming all messages between KAFKA topics
I’m using NIFI to stream messages between Kafka topics, using NIFI’s ConsumeKafka_2_ and PublishKafka_2_6 . The ConsumeKafka_2_6 defined with Offset Reset : earliest .
The source Topic x1 has 22 million messages but only 12 million messages streaming into the new destination Topic x2 .
Topic x1 and Topic x2 have the same definition.
Any idea why not all the 22 million messages streaming into the destination Topic x2 ?
One important thing,the cleanup.policy on topics is compact
Read more here: Source link