Hand detection and tracking methods

问题

So, guys, please help me with detecting/tracking hand for user who are sitting at the computer in front of computer(laptop) frontal camera. I've tried these methods:

• Colour based detection(I've detected the human face by opencv haar cascade face detection and extracted the skin HSV ranges. In the next I've found the objects with the skin colour. For example, the face I can remove by knowing face detection by haar cascade, but what about other human body parts and background objects with skin colour if I need only hand? How to make this algorithm be more stable for illumination?)
• Train own haar cascade classifier(I've trained my own cascade to detect hand using 3.5k positive and 4k negative photos. It took 3 days to train. The dataset is pretty rich(various hand configurations and orientations, light conditions, different backgrounds). It works not so bad but it's very slow because of I set scaleFactor=1.3 and minNeighbors=70. If I decrease minNeighbors false alarms will grow tremendously and small reactangles will cover the whole video frame. Training params: opencv_traincascade -data data -vec samples.vec -bg neg.txt -numStages 16 -minhitrate 0.999 -maxFalseAlarmRate 0.5 -numPos 3200 -numNeg 3900 -w 24 -h 24 -mode ALL -precalcValBufSize 1024`` -precalcIdxBufSize 1024

• Train LBP cascade classifier (The training was faster than haar cascade and detection works closer to real time but this detection method has a lot mishits) Training params: opencv_traincascade -data lbp -vec samples.vec -bg neg.txt -numStages 25 -minHitRate 0.999 -maxFalseAlarmRate 0.5 -numPos 3200 -numNeg 3900 -w 24 -h 24 -mode ALL -precalcValBufSize 4096 -precalcIdxBufSize 4096 -featureType LBP I tried the different values of numStages from 16 to 25.

• Camshift algorithm to track hand The source code is here http://pastebin.com/q5zK8cZt. How it works? Just need to mark 4 poins around detected object and this algorithm must track it and draw rectangle around. The problem is if I started to move my hand this rectangle starts to grow and cover the whole video frame. It looks like this algorithm works only for small objects (or the objects are locating long distance from camera)

Maybe I need to mix these methods or you will suggest another? Maybe I need to train neural network for example YOLO? I don't have wish to do it cause of it takes too long time and have to rent GPU-based servers.

回答1:

GPU servers? No, you don't: there is a web based backend for object recognition. If you want to use Yolo you will need to mark a huge image train set (Around 2000 per class). I can advise fetch images from here using script like that

(function(global) {
  const next = () => Array.from(document.querySelectorAll('.search-pagination__button-text'))[1].click();
  const uuid = () => Math.random().toString(36).substring(7);
  const sleep = (timeout = 5000) => new Promise((res) => setTimeout(() => res(), timeout));
  global.urls = [];
  global.next = () => next();
  global.start = async () => {
    for (let i = 0; i !== 81; i++) {
        window.scrollTo(0,document.body.scrollHeight);
        await sleep(5000);
        document.querySelectorAll('.search-content__gallery-results figure > img[src]').forEach(({src}) => global.urls.push(src));
        next();
        await sleep(5000);
    }
  };
})(window);

After that, you need to mark bounded boxes of objects in images. There is a online tool, which work right in your web browser

For training neural network follow this instruction, binaries also can be taken from here.

来源：https://stackoverflow.com/questions/42852813/hand-detection-and-tracking-methods