Please enable JavaScript.
Coggle requires JavaScript to display documents.
Kafka-Data-plane-replication Protocol - Coggle Diagram
Kafka-Data-plane-replication
Protocol
Core concepts
Log and Replicas
Log (Document/Book): Sequence of records/messages.
Replicas (Writers/Friends): Copies of the log for redundancy.
ISR (In-Sync Replicas): Up-to-date replicas.
Offsets and High Watermark
Offset (Page Number): Position in the log
High Watermark: Last committed offset.
Leader Election
Leader (Team Leader): Coordinates the log.
Control Plane (Coordinator): Manages leader election.
Data Plane (Writers): Handles data.
Leader Epoch (Version Number): Tracks leader changes.
Leader Election and Failover
Leader Failure
New leader selection from ISR.
No data loss
Temporary Decreased High Watermark
Offset_NOT_AVAILABLE error
Retriable errors.
Consumer retries.
Replica Reconciliation
Out-of-sync replicas
Follower fetch request
Offset discrepancy
Epoch verification
Follower Log Truncation
Under-replicated state.
Leader Balancing
Preferred Replica
Even distribution of leaders
Leader Imbalance
Periodic checks.
Configurable threshold.
Leader rebalance.