My experience is, we should use ajax after it works without it. For a couple of reasons.
First, if something breaks in the ajax, and you don't have it working without it, the site simply doesn't work. For example, a product list with pagination. It should work with the links alone, then use ajax when possible.
Second, for site indexing and accessibility. If it works without ajax, it's better.
And it's easier to break something (even if only for a few moments). A bad piece of code, an uncaught exception, an external library not loaded, a blocking browser extension,...
After everything works without ajax, its quite easier to add ajax. Just have the ajax catch the action, add ajax=1 and when returning the result, return only what you need if ajax=1, otherwise return everything.
In the product list example, I would only return the products and pagination html, and add to the correct div. If ajax stops working, the whole page is loaded and the customer sees the second page as it loads.
Ajax adds a lot of value to UX. If done right, the user gets a great feel when using the site, and better data usage because it doesn't load the whole page everytime.
But the question being "when not to use ajax", I would say, you should always count on it to improve UX but not rely on it for the site to work (as other users also mentioned). And nowadays we need both, great code and great user experience.