EDIT 4, 5 and 6
8 hours in, any more ideas are welcome :) Maybe this bug is already known and solved, but I get the behaviour I described in edit 2&
I had the exact same issue, and solved it by changing this option to true (it is at false by default) in my production.rb file.
# Disable Rails's static asset server (Apache or nginx will already do this)
config.serve_static_assets = true
From what I understand, setting this option to false makes sense if you are running under Apache or nginx, which was not my case.