I have configuration min.insync.replicas=2
and default.replication.factor=3
for my 3 node cluster.
If I try to produce when only one broker is up it was failed as I expected.
But If I try consume when only 1 broker is available the consumer is still able to consume messages. It seems min.insync.replicas=2
is not working for consumers. is it know behavior or I am missing anything ?
1 Answer
min.insync.replicas
specifies the minimum number of replicas that must acknowledge a write in order to consider this write as successful and therefore, it has an effect on the producer side which is responsible for the writes. This configuration parameter does not have any direct impact on the consumer side and this is why it does not affect Consumers, even if the number of alive brokers is less than the value of min.insync.replicas
.
According to the documentation,
When a producer sets
acks
to"all"
(or"-1"
),min.insync.replicas
specifies the minimum number of replicas that must acknowledge a write for the write to be considered successful. If this minimum cannot be met, then the producer will raise an exception (eitherNotEnoughReplicas
orNotEnoughReplicasAfterAppend
). When used together,min.insync.replicas
andacks
allow you to enforce greater durability guarantees. A typical scenario would be to create a topic with a replication factor of 3, setmin.insync.replicas
to 2, and produce withacks
of"all"
. This will ensure that the producer raises an exception if a majority of replicas do not receive a write.
-
Thanks for information, then how can we maintain consistency for consumer reads. is there any properties for that. Commented Jul 27, 2019 at 14:27
-
Consumer reads consistency is mainly guaranteed when pushing your committed offsets to the _consumer_offsets topic ( when your consumer does a commit). I know the default min isr for this topic is 1 ( for availability reason? ), but not sure with what kind of consistency it's pushed ( ack =1?)– YannickCommented Jul 28, 2019 at 10:10
-
OK, you can use offsets.commit.required.acks to control how many acks the group coordinator will await while producing to _consumer_offsets– YannickCommented Jul 28, 2019 at 13:54