kafka KRaft cluster + Metadata storage
we installed successfully Kafka Kraft cluster that include 3 controllers machines and 3 broker machines as the following
all machines with RHEL 8.x version
as we understand the folder – __cluster_metadata-0, is accumulates all the metadata changes of the cluster and its make sense that its related to controller cluster machines
but when we checked the brokers machines we found that this folder – __cluster_metadata-0
also exist on each of the broker machines
example
ls -1 /data/kafka/
Output:
bootstrap.checkpoint
__cluster_metadata-0
meta.properties
ls -1 __cluster_metadata-0/
Output:
00000000000000000000.index
00000000000000000000.log
00000000000000000000.timeindex
00000000000000000159.snapshot
00000000000000000288.snapshot
00000000000000000459.snapshot
00000000000000000630.snapshot
00000000000000000827.snapshot
00000000000000001032.snapshot
00000000000000001257.snapshot
00000000000000001419.snapshot
00000000000000001585.snapshot
leader-epoch-checkpoint
partition.metadata
quorum-state
so why Kafka architecture design reburied the metadata folder – __cluster_metadata-0 also on brokers machine ?
about the controllers machines its obvious that meta data folder – __cluster_metadata-0 should be part of controller machines but not clear why meta data folder exist also on brokers machines ?
related links – redpanda.com/guides/kafka-tutorial/kafka-without-zookeeper
Read more here: Source link

