5

I have configuration min.insync.replicas=2 and default.replication.factor=3 for my 3 node cluster. If I try to produce when only one broker is up it was failed as I expected. But If I try consume when only 1 broker is available the consumer is still able to consume messages. It seems min.insync.replicas=2 is not working for consumers. is it know behavior or I am missing anything ?

1 Answer 1

3

min.insync.replicas specifies the minimum number of replicas that must acknowledge a write in order to consider this write as successful and therefore, it has an effect on the producer side which is responsible for the writes. This configuration parameter does not have any direct impact on the consumer side and this is why it does not affect Consumers, even if the number of alive brokers is less than the value of min.insync.replicas.

According to the documentation,

When a producer sets acks to "all" (or "-1"), min.insync.replicas specifies the minimum number of replicas that must acknowledge a write for the write to be considered successful. If this minimum cannot be met, then the producer will raise an exception (either NotEnoughReplicas or NotEnoughReplicasAfterAppend). When used together, min.insync.replicas and acks allow you to enforce greater durability guarantees. A typical scenario would be to create a topic with a replication factor of 3, set min.insync.replicas to 2, and produce with acks of "all". This will ensure that the producer raises an exception if a majority of replicas do not receive a write.

3
  • Thanks for information, then how can we maintain consistency for consumer reads. is there any properties for that. Commented Jul 27, 2019 at 14:27
  • Consumer reads consistency is mainly guaranteed when pushing your committed offsets to the _consumer_offsets topic ( when your consumer does a commit). I know the default min isr for this topic is 1 ( for availability reason? ), but not sure with what kind of consistency it's pushed ( ack =1?)
    – Yannick
    Commented Jul 28, 2019 at 10:10
  • OK, you can use offsets.commit.required.acks to control how many acks the group coordinator will await while producing to _consumer_offsets
    – Yannick
    Commented Jul 28, 2019 at 13:54

Not the answer you're looking for? Browse other questions tagged or ask your own question.