Kafka configuration min.insync.replicas not working

Question

Its my early days in learning kafka. And I am checking out every kafka property/concept in my local machine.

So I came across this property min.insync.replicas and here is my understanding. Please correct me if I've misunderstood anything.

• Once a message is sent to a topic, the message must be written to at least min.insync.replicas number of followers.
• min.insync.replicas also includes the leader.
• If number of available live brokers( indirectly, in sync replicas ) are less than the specified min.insync.replicas , then producer will raise an exception failing to publish the message.

Following are the steps I followed to create the above scenario

1. Started 3 brokers in local with broker Ids 0, 1 and 2
2. created the topic insync and set min.insync.replicas to 2 using the following command

sudo ./kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 1 --topic insync --config min.insync.replicas=2

3. Describe the topic resulted in the following

Topic:insync PartitionCount:1 ReplicationFactor:3 Configs:min.insync.replicas=2 Topic: insync Partition: 0 Leader: 2 Replicas: 2,0,1 Isr: 1,2,0

At this point, I made sure the property I've provided is picked by kafka

4. I started sending messages and consuming them from terminal using following command

   Producer: ./kafka-console-producer.sh --broker-list localhost:9092 --topic insync --producer.config ../config/producer.properties

   Consumer: ./kafka-console-consumer.sh --zookeeper localhost:2181 --topic insync

At this point, I was able to send and receive messages successfully.

5. Bought down 2 brokers (0 and 2) and described the topic and resulted in following

Topic:insync PartitionCount:1 ReplicationFactor:3 Configs:min.insync.replicas=2 Topic: insync Partition: 0 Leader: 1 Replicas: 2,0,1 Isr: 1

At this point, the In Sync Replicas are just 1(Isr: 1)

Then I tried to produce the message and it worked. I was able to send messages from console-producer and I could see those messages in console consumer.

My Kafka version: kafka_2.10-0.10.0.0

following are the producer properties:

bootstrap.servers=localhost:9092

compression.type=none

batch.size=20

acks=all

I expected the producer to fail with NotEnoughReplicasException as mentioned in this.

public class NotEnoughReplicasException extends RetriableException

Number of insync replicas for the partition is lower than >min.insync.replicas

but it worked normally.

Am I missing something? How can I create the scenario?

*************** EDIT **********************

Instead of producing the messages from console producer, I tried to generate messages from java code. This time, I got the expected exception in the kafka broker. Although I expected it in the producer (java code). As this experiment is raising more questions, I've posted another question.

Answer 1

is acks set to "all"? if not, try setting it to all

Answer 2

1. I believe that error is for transactional producer, you may need to add this config:

   transactional.id=TID-TEST

2. if still not working, please check your replicator factor and min insync isr for the internal topic: __transaction_state