I have seen a similar issue when I open a jQueryUI dialog that has a TimyMCE editor in it. The scrollbars in the dialog were freezing.
The weird reason for it a URL HASH '#' in the end of the url. The URL HASH was added to the url because I forgot to 'return false;' in the click() event of the link that opened the dialog. Once I added the 'return false;' the event was not natively handled by the browser, no URL HASH was added and the scrollbars of the dialog worked as expected.