Can I throttle requests made by a distributed app?

Question

My application makes Web Service requests; there is a max rate of requests the provider will handle, so I need to throttle them down.

When the app ran on a single server

Answer 1

The N nodes need to communicate. There are various strategies:

• broadcast: each node will broadcast to everybody else that it's macking a call, and all other nodes will take that into account. Nodes are equal and maintain individial global count (each node know about every other node's call).
• master node: one node is special, its the master and all other nodes ask permission from the master before making a call. The master is the only one that know the global count.
• dedicated master: same as master, but the 'master' doesn't do calls on itslef, is just a service that keep track of calls.

Depending on how high do you anticipate to scale later, one or the other strategy may be best. For 2 nodes the simplest one is broadcast, but as the number of nodes increases the problems start to mount (you'll be spending more time broadcasting and responding to broadcats than actually doing WS requests).

How the nodes communicate, is up to you. You can open a TCP pipe, you can broadcats UDP, you can do a fully fledged WS for this purpose alone, you can use a file share protocol. Whatever you do, you are now no longer inside a process so all the fallacies of distributed computing apply.