I ran into this exact issue as well and found that it is only present with react-scripts@3.0.0, downgrading to react-scripts@2.1.8 resolved the issue without needing to mess with environment variables.
create-react-app my-app
cd my-app
npm install react-scripts@2.1.8
npm start