We have built a microservice architecture but have run into an issue where the messages going on to the bus are too large. (Discovered since moving to Azure Service bus as this only allows 256KB compared to RabbitMQ 4MB)
We have a design as the below diagram. Where we're struggling is with the data being returned.
An example is when performing a search and returning multiple results.
To step through our current process:
- Web client sends a http request to the Web Api.
- Web api then puts appropriate message on to the bus. (Web api responds to client with an Accepted response)
- Microservice picks up this message.
- Microservice queries its database for the records matching search criteria.
- Results returned from database.
- A SearchResult message is added to the bus. (This contains the results)
- Our response microservice is listening for this SearchResult message.
- The response microservice then posts to our SignalR api.
- SignalR Api sends the results back to the web client.
My question is how do we deal with large results sets when designed in this way? If it's not possible how should the design be changed to handle large results sets?
I understand we could page the results but even so one result could be over the 256KB allowance, for example a document or a particularly large object.
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…