Controlling mouse pointer through camshift algorithm of OpenCV (or how mouse basically functions)

Question

I have written a program in C# using EmguCV (wrapper of openCV). The program tracks an object using camshift algorithm. A rectangle is drawn around the object. The cursor is moved by center of the rectangle. Input is taken from a webcam.

Initially the problem was that the cursor couldn't cover the entire screen. Its movement was limited to frame size. And so i applied a multiplication factor:

screenwidth/framewidth for movement in X direction.

screenheight/frameheight for movement in Y direction

With this the mouse covers entire area. But the mouse movement is not smooth any more. I am unable to point to two icons closely situated. How can I make mouse movement smooth while covering the entire screen, just as it happens with a real mouse?

Answer 1

To point out the obvious: the non-smoothness of your mouse implementation comes from the fact that the rectangle given by camshift is only accurate to the accuracy of one frame pixel, so the smallest movement possible will be screensize/framesize rounded to the nearest screen pixel.

If that is the case, some kind of pointer acceleration could be applied, just like it is done when real, low-quality mice are used (of course, in case of multiple thousand dpi laser mice, there is no need for such thing). Basically, the distance the cursor moves on screen is not the distance taken by the pointer input (in this case, camshift rectangle displacement), but a cleverly chosen function of it. So, using an acceleration function f(x), the steps of moving the pointer will be like this:

1. Calculate vector of pointer input displacement, let that be denoted by v.
2. Calculate the corresponding unit-length vector, let that be denoted by u.
3. The on-screen pointer displacement is v'=f(|v|) * u

I'd chose f(x) in a form like beta * e^(alpha * x - 1), where 0 < alpha and 0 < beta <= 1 are parameters which should be empirically chosen.

Basically, any function will do it that has a derivate of 1 or less at 0 (allows you to use the full accuracy of the input for precise cursor movements), goes to infinity as x increases (large movements should correspond to large movements of the cursor), is monotonically increasing and has a monotonically increasing first derivate. Edit: It is also needed that the acceleration function has a value of 0 at 0, otherwise very strange movements will happen. :)

It is also desirable to have f(framewidth) = screenwidth so that moving the tracked object across the frame results in the cursor being moved across the screen. An exponential formula is quite pleasing to work with, but using a quadratic or higher degree polynomial may turn out to be computationally simpler, depending on what performance requirements are there...