I have a single page application, having sensitive content, and needs to be secured. This question is specific with securing against XSS and CSRF attacks.
Explan
CSP is a rather new concept for me, but from what I know, my answer is: use them all. I'll try to elaborate.
I was recently thinking about a very similar problem - but the difference in my scenario has a greater impact than your case.
Specifically I was thinking about an OLTP application which used cacheable content for all the HTML and AJAX/SJAX + JSON for retrieving transactional data. I've yet to consider in depth wither to use a conventional POST or AJAX/SJAX for sending the transactional data back to the server.
The advantage of this approach for me (over a conventional HTML / GET / POST OLTP) is that everything apart from the transactional data becomes cacheable allowing for optimal capacity, and based on the expected usage scenario a performance benefit.
It's also a gateway to implementing PJAX on the site thereby eliminating the startup cost retrieving content from the local cache and parsing. But I'm waffling about stuff which is not really relevant to your question.
Your non-cookie approach similarly isolates the cacheable content from the data - this is not strictly true of all single page applications, but with fewer page level transitions the net benefit is not as great.
As you say, CSP reduces the likelihood and impact of a XSS attack, but it does not eliminate it - in the case where the script is stored on the victim site and replayed to visitors. Despite there being no real "cross-site", this is still described as a type of XSS attack. And it still allows a request forgery to be directed at the same-origin.
Hence unless you can be completely confident that your CSP will never allow unsafe-inline, I think you still need some sort of CSRF protection in there. The Microsoft solution is neat in that it is stateless - but is reliant on the page of your single page application being non-cacheable (which eliminates the performance benefit).
XSS
Is a vulnerability in your application that allows attackers to have your regular visitors unwarranted execution of 3rd party controlled javascript.
The protection form that has to come from filtering all user supplied input (even if it's stored in a database) before outputting it in the context of the output by escaping the needed characters that create the problem in the given context.
Authentication tokens are one of the many potential targets of a XSS attack, and storing them as http_only helps in protecting them, but it is by far not enough to keep attackers out.
CSRF
Cross site request forgery is essentially a vulnerability in your application where an authorised user follows a link on e.g. a 3rd party website that makes them unknowingly execute a change on your application.
The traditional protection involves generating a "key" that you output in a hidden field in every form that can make a change (and convert every button that makes a change in a form with said protection). The key has to change often enough so that an attacker cannot know them, guess them etc. while it stays long enough the same that the user has time to fill out the form. Typically they can be generated along with a session, but you do NOT store them on the client, only output them in every form and check they are there before accepting the input of a post.
Using local storage to store that CSRF key (if you are sure all your visitors support it) would be a possibility but you add problems if the assumption that all browsers support it is not true (they'll all generate CSRF violations)
HTTP_ONLY
Means the browser is instructed to not let javascript access to this cookie. This helps somewhat to minimise access by XSS attacks to this cookie (but not to other things of interest to the attacker)
Secure
Means the browser is instructed to only send this cookie to the server when using a https connection (and not also send it on a non-https connection to the same server (where it could be intercepted along the way).
putting it together
Use the authentication token as a cookie, with https_only and secure options. If you are sure all your clients are capable of local storage, you can add a CSRF key to local storage and add javascript that extracts it and sends it to the server in every form and on every button that makes a change. If you keep both keys separate you get the best of both worlds. BUT: you still have to check the CSRF key to be present and valid on every request that changes anything.
Personally I feel this is too soon, and just using a traditional CSRF key in every form generated by the server is for now easier than having to rely on assumptions about browsers and/or providing fallback mechanisms that do the same as the old thing and having to maintain 2 methods.
CSP
CSP is nice, but by far not in every browser out there. And if you rely on it to stop XSS, I'd consider it as a "belt and suspenders" approach, not as the only solution. The reason is simple enough: someday you get too much load and decide to a a CDN ... oops now the CDN needs to be allowed to load images, scripts etc. and now the XSS door is open to any other user of that CDN ... Maybe the one opening that door didn't even consider XSS as something they would need to worry about.
Moreover context-sensitive output filtering isn't that hard to write and and it makes it much safer to output just about anything anywhere and have the filters take care of encoding (e.g. you URLs would get urlencoded where needed, your html tags' attributes get properly escaped and your text can include & and < without having to worry about anything anymore.