Kafka data got compacted/deleted not respecting the configuration

We are using Strimzi to manage our Kafka cluster in Kubernetes.

Below is our Strimzi configuration for our Kafka cluster (removed the SSL and authorization confs below):

[kafka@k8s-kafka-2 kafka]$ cat /tmp/strimzi.properties
##########
# User provided configuration
##########
log.message.format.version=2.6
log.retention.hours=-1
offsets.retention.minutes=525600
offsets.topic.replication.factor=3

We are using Infinite log retention to never delete data from our Kafka topics.
We also have a default cleanup policy (which is delete and not compact)

Today, our topics oldest offset moved from 0 to high numbers. We lost a lot of earlier messages due to this.

We are trying to figure out why it happened when we are using infinite retention (log.retention.hours=-1).

Any help is appreciated. Let me know if adding which details can help you help me.

Why could Kafka compact the topics or move its offset from 0 to higher numbers when the retention is infinite?

Read more here: Source link