Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
1.3k views
in Technique[技术] by (71.8m points)

rabbitmq - Message loss issues reported in production

We have received multiple message loss issues in PROD recently.

Upon investigating such issues we have so far concluded that

  1. Publishers are sending messages to RabbitMQ using messaging framework component

  2. We have seen only some of the messages get published into RabbitMQ queues and have seen the same using RabbitMQ queue REST API by monitoring the "publish" and "deliver" fields in message_stats before sending the messages and after sending the messages.

in a recent use case if say 28 messages were sent the "publish" and and "deliver" that give the cumulative values got increased by only 20 and 8 messages were lost.

Also the logging implemented in messaging framework confirmed this behavior .The messaging framework recorded the logging of 28 incoming messages wheras the Post processor on the receiving side recorded only loggings for 20 messages.

  1. Since we are using Spring AMQP for publishing the messages , so in the messaging framework we are not having control to further debug what is going wrong between Spring AMQP and RabbitMQ.

There was one more observation that we did the restart of the application and it solved the issue once in one particular use case and we could receive the logging of all messages on sender as well as receivng side.

Need some suggestion over here .Also recently we have rolled out the pause minority configuration in PROD without publish confirm. Can it be the cause of message loss. We will investigate the logs further though regarding this.

Any help would be beneficial over here

Regards,

Apoorv

question from:https://stackoverflow.com/questions/65841974/message-loss-issues-reported-in-production

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)
Waitting for answers

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...