Line search fails in training ksvm prob.model

Question

Following up from Invalid probability model for large support vector machines using ksvm in R:

I am training an SVM using ksvm from the kernlab package in R. I want

Answer 1

I do not understand the behavior of the optimizer. if max iteration is reached, no problem. but if step is lower than min_step it calls .SigmoidPredict which does not return A and B. I do not think that the solution is to decrease min_step, but not to call .SigmoidPredict, so I commented it out. btw, I do not understand why they do not use glm to estimate A and B.

here's a repository based on the latest source from cran with the call to SigmoidPredict commented out.

devtools::install_github('elad663/kernlab')

Answer 2

It seems to me that the problem occurs randomly. Thus, I circumvented the problem by fitting the ksvm model as many times until it worked.

stop.crit = 1
while (stop.crit <= 10) {
    stop.crit = stop.crit + 1
    MOD = ksvm(...)
    tryCatch(PRED = predict(...), error = function(e) e)
    if (exists("PRED") == TRUE) stop.crit = 11
}

Answer 3

Looking at the source code, this is the line that throws that error.

It's on the method .probPlatt using the Newton method to optimize the function, in this case Platt's scaling. If you check line 3007 though you'll see some parameters pertaining to the method.

One of such parameters is minstep basically the minimal numeric step the method should keep trying to optimize the function. You see, this is exactly the condition of the error in line 3090: if (stepsize < minstep). So, basically, the function is not converging, even when reaching the minimum step size.

You can try changing minstep to lower values to circumvent it. Alexandros even commented these parameters should probably be in the interface.