Python: Find Amount of Handwriting in Video

Question

Do you know of an algorithm that can see that there is handwriting on an image? I am not interested in knowing what the handwriting says, but only that there is

Answer 1

I don't think you really need the code in this case and it would be rather long if you did. But here's an algorithm to do it.

Use OpenCV's EAST (Efficient Accurate Scene Text detector) model at the beginning to establish the starting text on the slide. That gives you a bounding box(es) of the initial percentage of the slide covered with slide text as opposed to handwritten explanatory text.

Every, say 1-5 seconds (people don't write all that fast), compare that baseline image with the current image and the previous image.

• If the current image has more text than the previous image but the initial bounding boxes are NOT the same, you have a new and rather busy slide.

• If the current image has more text than the previous image but the initial bounding boxes are ARE the same, more text is being added.

• If the current image had less text than the previous image but the initial bounding boxes are NOT the same, you again have a new slide -- only, not busy and with space like the last one to write.

• If the current image has less text than the previous image but the initial bounding boxes are ARE the same, you either have a duplicate slide with what will presumably be more text or the teacher is erasing a section to continue, or modify their explanation. Meaning, you'll need some way of addressing this.

When you have a new slide, take the previous image, and compare the bounding boxes of all text, subtracting the boxes for the initial state.

Computationally, this isn't going to be cheap (you certainly won't be doing this life, at least not for a number of years) but it's robust, and sampling the text every so many seconds of time will help.

Personally, I would approach this as an ensemble. That is an initial bounding box then look at the color of the text. If you can get away with the percentage of different color text, do. And when you can't, you'll still be good.