Semantics of acks=1 min.insync.replicas=1 retries=10

Question

I'm struggling to figure out what kind of guarantees a system with this producer configuration provides

acks=1 
min.insync.replicas=1 
retries=10

The first two parameters basically give me at-most-once guarantees since the leader can write to local storage, reply to the producer and then die before replication to the followers.

That part is clear but the retries would mean that when there are network issues between the producer and the cluster the leader may write to the local disk and fail to confirm this fact to the producer. After that, the producer would retry the writing and duplicate the message.

Is my reasoning correct that this setup technically provides no specific delivery guarantee? Or is there a process on the cluster that prevents the latter from happening?

Answer 1

You are correct, the first two configuration properties provide "at most once" guarantee only.

For "at least once" guarantee, you need to change acks to all (or -1). This would only send an acknowledgment when all members of the ISR (leader and followers) have received the record.

Even with acks=all, in some cases this will still only guarantee "at most once" delivery IF there is only 1 replica in the ISR due to failures.

To ensure acks=all always guarantees "at least once" delivery, you need to set min.insyc.replicas to at least 2 (assuming you have more than 1 broker in your cluster). This would mean that at least 2 replicas must have received the record for the producer write to be considered successful. If min.insyc.replicas is not met, it will throw a NotEnoughReplicas exception.

On to the second part of your question. Yes, you can receive duplicate messages with this configuration. To prevent duplicates, you can set enable.idempotence=true.

However, when setting enable.idempotence=true, this requires max.in.flight.requests.per.connection to be less than or equal to 5, acks must be all and retries have to be greater than 0.