I am aware there are a lot of questions about running Docker on windows, however this question is about running the brand new Docker for Windows, on Windows.
In my case I am
Here's a new twist.
The last Windows 10 update (Fall Creators Update, 2017) has a new "feature". It automatically starts any applications that were running when you last shutdown.
This reconstitutes Docker for Windows in a bad state. That made it appear those ports were in use by something else - it was the ghost of itself. This explained why those ports were still in use even though I stopped/started my containers and even reboot!
The solution in this case is to simply restart Docker daemon.
To prevent this after the next shutdown, don't use the shutdown button. Type this instead:
shutdown /s /t 0
This bypasses the new feature.
See the answer from Jason[MS] in this thread:
https://answers.microsoft.com/en-us/insider/forum/insider_wintp-insider_perf-insiderplat_pc/programs-autostart-after-boot-in-windows-10-fall/09dd8d3e-7b36-45d1-9181-6587dd5d53ab
Here's one guy's workaround (from the end of this thread - haven't tried it myself):
http://www.icttoolbox.nl/info/stop-windows-10-creator-fall-reopening-programs-reboot/