So I did some digging in the AWS forums and this does seem to be a known "issue". Looks like there is a a fix:
in .ebextensions/wsgihacks.config:
files:
"/etc/httpd/conf.d/wsgihacks.conf":
mode: "000644"
owner: root
group: root
content: |
WSGIPassAuthorization on
Original thread: https://forums.aws.amazon.com/message.jspa?messageID=376244