Line search fails in training ksvm prob.model

问题

Following up from Invalid probability model for large support vector machines using ksvm in R:

I am training an SVM using ksvm from the kernlab package in R. I want to use the probability model, but during the sigmoid fitting I get the following error message:

line search fails -1.833726 0.5772808 5.844462e-05 5.839508e-05 -1.795008e-08 
-1.794263e-08 -2.096847e-12

When this happens, the resulting value of prob.model(m) is a vector of all probabilities, rather than the expected parameters of a sigmoid function fitted over these probabilities. What causes this error and how can I prevent it? Searching for the error message yielded no results.

Reproducible example:

load(url('http://roelandvanbeek.nl/files/df.rdata'))
ksvm(label~value,df[1:1000],C=10,prob.model=TRUE)->m 
prob.model(m) # works as it should, prints a list containing one named list

# the below, non-working problem, unfortunately takes an hour due to the large
# sample size
ksvm(label~value,df,C=10,prob.model=TRUE)->m # line search fails  
prob.model(m) # just a vector of values

回答1:

Looking at the source code, this is the line that throws that error.

It's on the method .probPlatt using the Newton method to optimize the function, in this case Platt's scaling. If you check line 3007 though you'll see some parameters pertaining to the method.

One of such parameters is minstep basically the minimal numeric step the method should keep trying to optimize the function. You see, this is exactly the condition of the error in line 3090: if (stepsize < minstep). So, basically, the function is not converging, even when reaching the minimum step size.

You can try changing minstep to lower values to circumvent it. Alexandros even commented these parameters should probably be in the interface.

回答2:

It seems to me that the problem occurs randomly. Thus, I circumvented the problem by fitting the ksvm model as many times until it worked.

stop.crit = 1
while (stop.crit <= 10) {
    stop.crit = stop.crit + 1
    MOD = ksvm(...)
    tryCatch(PRED = predict(...), error = function(e) e)
    if (exists("PRED") == TRUE) stop.crit = 11
}

回答3:

I do not understand the behavior of the optimizer. if max iteration is reached, no problem. but if step is lower than min_step it calls .SigmoidPredict which does not return A and B. I do not think that the solution is to decrease min_step, but not to call .SigmoidPredict, so I commented it out. btw, I do not understand why they do not use glm to estimate A and B.

here's a repository based on the latest source from cran with the call to SigmoidPredict commented out.

devtools::install_github('elad663/kernlab')

来源：https://stackoverflow.com/questions/15895897/line-search-fails-in-training-ksvm-prob-model

标签

svm

kernlab