I am using Jekyll as a static generator for a website (not a blog), and I want to have an automatically generated list of all pages on my index page. Specifically, I want to
You should differentiate between pages and posts (articles). Listing all posts sorted by category is not a problem at all. You can loop through site.categories. It contains the category name and a list of all posts in that category.
Listing all pages is possible, too. You can loop through site.pages. But a page does not belong to a specific category (only posts do).
When I take a look at your posted example, using categories on posts and then looping through site.categories seems to be the way to go. It will get you exactly the desired output.