Python.h: No such file or directory on Amazon Linux Lambda Container

Question

I am trying to build this public.ecr.aws/lambda/python:3.6 based Dockerfile with a requirements.txt file that contains some libraries that need gcc/g++ to build. I\