Combining features of all videos using Python

Question

Let\'s assume I have 20 videos. They are of the same scene, dimensions and from the same camera. Let\'s assume one of those twenty videos has a person walking across. All the ot

Answer 1

Thanks to the hint from @Stephen Meschke, I got it working and realized it's pretty good once you do it correctly its not really a good approach to do what I wanted to do. The difference between "background" and "foreground" isn't really good.

Anyway, this is my code. If anyone sees ways to improve it, please let me know:

"frame" is the frame from the new video. "frame_b" is the blended video that gets created each iteration of video processing.

kernel_clean = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(3,3))
kernel_fill = np.ones((20,20),np.uint8)

# get foreground objects from new frame
frame_mask = fgbg.apply(frame)
# clean noise
frame_mask = cv2.morphologyEx(frame_mask, cv2.MORPH_OPEN, kernel_clean)
# fill up foreground mask better
frame_mask = cv2.morphologyEx(frame_mask, cv2.MORPH_CLOSE, kernel_fill)

# remove grey areas, or set detectShadows=False in the extractor, which I learned later. However, removing shadows sometimes causes gaps in the primary foreground object. I found this to produce better results.
indices = frame_mask > 100
frame_mask[indices] = 255
# get only foreground images from the new frame
foreground_a = cv2.bitwise_and(frame,frame, mask=frame_mask)
# clear out parts on blended frames where forground will be added
frame_mask_inv = cv2.bitwise_not(frame_mask)
modified_frame_b = cv2.bitwise_and(frame_b, frame_b, mask=frame_mask_inv)
merged_frame = cv2.add(modified_frame_b, foreground_a)