erlang - how to limit message queue or emulate it?

Question

Right now I am doing Process ! Message, but as i googled a bit, a message queue size is only limited to memory. I have a tree of processes where leaves generate messages and

Answer 1

There are no built-in mechanisms to limit the size of the message queue for a process.

The usual solution to this problem in erlang is to introduce a flow control protocol between the producer and the consumer. This can be as simple as the sender waiting for a continue reply before sending the next message. You can invent more complicated flow control protocols (windowing, ...), but often send/wait-reply will do. The gen_server:call/2 protocol is a good request-response protocol and can be re-used by looking at the code for gen_server and gen:call - it takes care of a lot of the edge cases that are possible.

Another approach is to pull messages through the system instead of pushing them - in this case the receiver asks the sender for a message when it's ready. Though if you have external producers that you can't rate-limit or buffer sufficiently, then you may not have this option.

Answer 2

RabbitMQ implements this in the form of Credit Flow. More info on this blog post: http://videlalvaro.github.io/2013/09/rabbitmq-internals-credit-flow-for-erlang-processes.html

Answer 3

The classical way of handling your second case is to include the Pid of the sending process in the message. This is very common for erlang messages which usually have the structure similar to {SendingPid,Data}. This allows the receiving to process both to see who sent the message and by using pattern matching within the receive to select from which process it wishes to receive messages.

This is how erlang can multiplex messages from many different senders on one message queue and not be forced to handle all possible messages everywhere.