Please enable JavaScript.
Coggle requires JavaScript to display documents.
Kafka - Coggle Diagram
Kafka
Brokers
Topic
Partition
message
append only
order guaranteed within partition only not topic
offset
zookeeper coordinates between the brokers
receive messages from producers
assign offsets
save on storage disk
response to fetch requests
response with messages to consumers
multiple brokers for replication
list
describe
Why Kafka ?
Multiple Producers
Multiple Consumers
Scalable
Retention
High Performance
Kafka Use Cases
Messaging Systems
Stream Processing
IoT World
Event Driven Decisions
Data Integration
Replication and fault-tolerance
Kafka Consumer
specify
Topic
group_id
Broker: host and port
auto_offset_reset
earliest
latest
consumer_timeout_ms
Consumer Groups
Scalability
offsets
rebalance
commit offset
consumer.seek
read from specific offset
Kafka Producer
specify
broker: host and port
Topic
ack
1
0
all
check if the message has arrived to the broker
key
messages with the same key go to the same partition
message value
max.in.flight.requests.per.connection
set to 1 guarantee order
messages are sent in batches