Multiple instance of a Service - SQS consumer

Question

I have a Service called DataProcessService which is deployed with multiple instances in the service for scaling.

The DataProcessService has SQS consumer which receives a message from the SQS topic, once the process is done the service deletes the message from the queue.

I have enabled SQS message visibility-timeout as 15 mins

Now the problem I have is: one of the instances of DataProcessService receives a message and do the processing and takes more then 15 mins, meanwhile other instance receives the same message due the 15mins visibility-timeout.

How to make sure, if an instance is still processing the message, and make sure another instance does not pick the message to process

Answer 1

I have faced similar issues too, So my way of dealing with it is. I will be calling each instance as a process because i am assuming the same business logic being performed from both instances right? Correct me if my understanding is wrong.

1. Remove the message from the SQS explicitly as soon as the process received it.
2. once you read it in service make sure you logit properly so that you will have a track of message received + deleted.
3. Now, lets say two processes received the same message at exactly same time, but you will let you code decide only based on if it successfully deleted the SQS message. Otherwise your process should not perform any business logic, to avoid duplicity.

Hope this works for you.