HOG Person Detector: False Positive detections on background subtracted images

Question

I am working on a project which requires detection of people in a scene.

Initially after running the HOG detector on the original frames a particular background object was being detected as a person on all the frames, giving me 3021 false positive detections.

So I took the logical step to remove the static background by applying a background subtracter (BackgroundSubtractorMOG2) to all the frames. The resulting frames looked like this:

Then these mask images were added (using bitwise_and) to the original image so the white pixels are replaced the pixels constituting the person.

Sample:

Then I ran the HOG detector on these images which gave the results like this:

As you can see there are a lot of false positive detections for some reason. I thought doing background subtraction will give me better results than using HOG on the original images.

Can someone please tell me why there are so many false positives in this method? And what can be done to improve the detection on background subtracted images?

Answer 1

The problem is that you changed the nature of your image by removing the background. So, the HOG detector was trained with normal images, without artificial black pixels, and now you are feeding it artificially altered images, so it is normal that it will perform in an weird way (still don't understand that detection at the top of the image though..)

If you want to use HOG detector on top of the background subtraction, you should train the HOG classifier with features taken from the background subtracted images.

One thing you can try (if this doesn't kill the performance of you application), is to use HOG detector on both images, with and without background, and accept only detections that overlap significantly on both, this may remove some false positives from both images.

PS: HOG was specially designed to work on raw images by detecting strong edges and test them against an SVM model. By removing background, we are creating artificial edges that kinda defeat the purpose of using HOG. But I think you can use it to remove false detections by doing what I suggested in the previous paragraph.