I have been building a prototype page over the past few months that uses a lot of SVG and has a lot of elements in general. There is also a ton of data being processed both
1) Surely your thousands of event listeners could be consolidated through event bubbling. Using a single, master event handler with different subroutines for different event targets would be more performant than numerous specific handlers.
2) "until Web Workers are supported by IE there isn't much of an elegant solution."
Au contraire, mon frère: freezing the browser can be mitigated by doing the processing in smaller chunks (I'd try to keep it under 100ms for each callback, if at all possible) and executing the next step after a timeout, which gives the browser a chance to update its state and process user input.
3) If you have a huge number of elements, it sounds like the HTML5 Canvas element is a more appropriate solution than SVG.
4) "My code is far from optimized"
Algorithmic optimizations make all the difference when you're pushing the limits like this.
5) DOM access is very expensive, so huge gains can be made by cleverly minimizing the number of DOM operations. Be sure you're not touching each element, one at a time. Better to reconstruct the whole mess and replace it all in one DOM manipulation.