JQM just helps you build a web page/application that is optimized for mobile devices, however it is still just a web page. On iOS you can pin a web application to the home-screen which opens up a separate browser instance when clicked/tapped as well as provide some meta tags that can make your web application appear more like a native app (for example you can hide the status bar at the top of the screen and provide a splash screen). On android however (at least on ICS, don't know about Jelly bean), while you can pin web application to your home-screen it just acts as a shortcut and opens it up as another tab in your browser. I'm not sure what options you have for web apps on windows phones.
Phonegap just builds a native app wrapper with a webview (a browser instance) for your web application and gives you access to the devices API.
Basically the two are not mutually exclusive, you can build your web application using JQM and still wrap it up in phonegap (you'll probably still run into whatever difficulties you ran into before), or you can skip the phonegap part, but your web application will still be just a web application, i.e a web page (that is more apparent on android then iOS).