Supprt Vector Machine works in matlab, doesn't work in c++

Question

I\'m writing an application that uses an SVM to do classification on some images (specifically these). My Matlab implementation works really well. Using a SIFT bag-of-words

Answer 1

There is nothing magical about the Matlab version of the libraries, other that it runs in Matlab which makes it harder to shoot yourself on the foot.

A check list:

1. Are you normalizing your data, making all values lie between 0 and 1 (or between -1 and 1), either linearly or using the mean and the standard deviation?
2. Are you parameter searching for a good value of C (or C and gamma in the case of an RBF kernel)? Doing cross validation or on a hold out set?
3. Are you sure that your're handling NaN, and all other floating point nastiness? Matlab is very good at hiding this from you, C++ not so much.
4. Could it be that you're loading your data incorrectly, reading a "%s" into a double or something that is adding noise to your input data?
5. Could it be that libsvm/dlib expects the data in row major order and your're sending it in in column major (or the other way around)? Again Matlab makes this almost impossible, C++ not so much.
6. 32-64 bit nastiness one version of the library, executable compiled with the other?

Some other things:

1. Could it be that in Matlab you're somehow leaking the class (y) into the preprocessing? no one does this on purpose, but I've seen it happen. If you make almost any f(y) a feature, you'll get almost 100% everytime.
2. Sometimes it helps to verify that everything is numerically identical by printing to file before training both in C++ and Matlab.

Answer 2

i'm very happy with libsvm using the rbf kernel. carlosdc pointed out the most common errors in the correct order :-). for libsvm - did you use the python tools shipped with libsvm? if not i recommend to do so. write your feature vectors to a file (from matlab and/or c++) and do a metatraining for the rbf kernel with easy.py. you get the parameters and a prediction for the generated model. if this prediction is ok continue with c++. from training you also get a scaled feature file (min/max transformed to -1.0/1.0 for every feature). compare these to your c++ implementation as well.

some libsvm issues: a nasty habit is (if i remember correctly) that values scaling to 0 (zero) are omitted in the scaled file. in grid.py is a parameter "nr_local_worker" which is defining the mumber of threads. you might wish to increase it.