I have a jQuery Mobile website that works great on normal browsers, but it fails when using Opera M
You can use Opera Mobile Emulator together with an older version of Opera that has the Dragonfly inspector (replaced in newer versions with Webkit Inspector) and remote debug your website.
Could it be because Opera Mini has a limited amount of JavaScript support?
Your best bet would be to rewrite what you need for Opera from the current support documentation guidelines from Opera Mini: web content authoring guidelines, JavaScript support.