I have following setup:
The application https://app.domain.de
is our production environment and is automatically forwarded to use HTTPS. All works fine here
On the main domain, you can remove the includesubdomains option of your HSTS header, so it will not redirect the sub domain.
However, this is not the most secure solution. To be effective, it's better to set HSTS+includesubdomains on all your domains and subdomains (or an attacker car fake the domain "http://secure.yourdomain.com" for example).
So the most secure solution is to use a self-signed certificate (or a real one) for your dev domains and import it before in your browsers.