This seems to be a bug in MSMF backend of opencv. You can see more details in this issue.
I don't think this problem exists on Linux platforms. So I am providing the solution for windows.
Windows only solution
For windows platform, you can change the backend to something else (most preferably DirectShow backend. For this, add to your VideoCapture
like below:
captureDevice = cv2.VideoCapture(0, cv2.CAP_DSHOW) #captureDevice = camera
This works for OpenCV>=3.4
.