I wouldn't mix a Servlet/JSP tech with an SPA (single page app....driven by angular in your case). What you can do is use a tool like SpringMVC (or Jersey) that has a single JSP which is your angular driven SPA. That way you have lots of control over the initial HTML/JS/CSS payload in the initial response. Once that "app" is loaded, all it's communication with the server is done via XHR calls ($http or $resource in angular). Spring makes this pretty simple to create using @Controller and giving you all the flexibility you need... things like spring security and dependency injection.
I've been working in my spare time on something simple just like this to help java server side people get into Angular.
the idea is that you'd have a controller that returns a JSP when you make a GET request to
http://your.site.com/contextRoot
That page would have the JS/CSS links to load Bootstrap/JQuery/Angular/Whatever... From there the angular router would kick in and your URL might end up like this
http://...../contextRoot#home
All communication between the JSP running angular on the client and the server is done with $http calls and you can make controllers in Spring to handle all this.
I'll post back here once I have my "Springular" app available