问题
In my project, I have to design a real time notification system. And I did it as the image below.
You can see that I used Kafka as a queue messaging system, and NodeJS to build Websocket Server and Kafka Consumers. Producers will collect the notification data and push it to Kafka. Consumers will read and process the data from Kafka and push it to client via websocket if that data belongs to that user.
With the architect above, Each online user will open a Websocket connection and create a new Consumer. It means that if there are 100K user online, we have to have 100K Consumers.
So my question is the design above is a correct design for a real time notification system? Do you have any different idea? Is there any issue if I have 100K Consumer?
EDITED
It should not open too many consumers. It is able to work with a consumer and websockets
回答1:
Your design is correct, if you want to be connected with each of your client all the time then you will have to keep the same number of client-websocket as there are number of clients; but consumer doesn't have to be increased with respect to the clients connected; since they can be shared among the different client-websockets.
It depends on your application and the way you want to handle it.
来源：https://stackoverflow.com/questions/33130276/real-time-notification-with-kafka-and-nodejs