Producers, Record Keys, Acknowledgements, Batching, and Retries
Learn how Kafka producers send data efficiently and reliably, and how configuration choices affect durability, throughput, and ordering.
Inside this chapter
- What a Producer Does
- Important Producer Concepts
- Producer Example in Java
- Acks and Reliability
- Batching and Compression
- Operational Example
Series navigation
Study the chapters in order for the clearest path from Kafka basics and local setup to stream processing, platform operations, cloud usage, and advanced event-driven architecture thinking. Use the navigation at the bottom to move smoothly through the full tutorial series.
What a Producer Does
A Kafka producer publishes records to topics. At first glance this seems simple, but producers are where many important reliability choices begin: partition selection, record keys, batching, compression, retry behavior, and acknowledgement settings.
Important Producer Concepts
- Topic selection
- Key selection for partition routing
- Value serialization
- Acknowledgement mode
- Retry behavior
- Idempotence and delivery guarantees
Producer Example in Java
ProducerRecord<String, String> record =
new ProducerRecord<>("orders", "order-101", "{\"status\":\"CREATED\"}");
producer.send(record);The key order-101 helps preserve ordering for that order’s event stream.
Acks and Reliability
The acks setting controls how many broker acknowledgements the producer waits for before considering a send successful. Stronger acknowledgement settings improve durability but may affect latency.
Batching and Compression
Kafka producers can batch records for throughput efficiency. Compression can reduce network and storage cost. Advanced users should understand these as throughput tools, not magical defaults to ignore.
Operational Example
If a checkout service produces order events too aggressively with weak acknowledgement settings, the business may lose critical events during failures. Producer configuration directly affects trust in the system.