Google\'s GMail service does it because it integrates Google Talk -- and Etherpad (now typewith.me) made famous the system which is used by, for example, Google Wave. All su
As everyone says.. AJAX.
The client keeps on asking the server, after say 30 secs if there is anything new for it to do. Also, you can set the timeout value on an ajax request. keep the time out a bit high.. and the server replies whenever there is something new.
There is no way that the server can other wise ask the client to load some data.
If you are thinking of implementing something on the same lines, look up strophe.js which is an XMPP js-library