Support Vector Machine for Java?

Question

I\'d like to write a \"smart monitor\" in Java that sends out an alert any time it detects oncoming performance issues. My Java app is writing data in a structured form

Answer 1

Weka is a popular machine learning/data mining package in Java. This book http://guidetodatamining.com/ might be useful. It does not really address SVM's but it certainly has good classification algorithms, and it is certainly not at all esoteric.

Answer 2

Perhaps Apache Spark MLlib will help you:

The linear SVM is a standard method for large-scale classification tasks. It is a linear method as described above in equation (1), with the loss function in the formulation given by the hinge loss:

L(w;x,y):=max{0,1−ywTx}.

By default, linear SVMs are trained with an L2 regularization. We also support alternative L1 regularization. In this case, the problem becomes a linear program.

The linear SVMs algorithm outputs an SVM model. Given a new data point, denoted by x, the model makes predictions based on the value of wTx. By the default, if wTx≥0 then the outcome is positive, and negative otherwise.

Answer 3

A "smart monitor" you describe is exactly time-series classification.

There are many classification algorithms. They all basically take an matrix, where the rows are observations and the columns are "features" that somehow describe the observation, and a label vector of length rows that is valued either 0 or 1. In your problem an observation might be a minute sample, and your label vector will be valued 1 for the time periods that are experiencing performance issues and 0 otherwise.

Implicit in this definition is the need to resample your data(using the mode/median/mean if necessary) such that each observation is defined evenly, such as seconds or minutes or hours.

Generating features is the crucial part. I'd probably start with 2 features, the raw values and the (once) differenced values between observation x_i and x_i-1. We'll define these for a lag of 2. Technically making this 4 features. Each feature can't look into the future. Each feature must represent the same thing for each observation.

For example consider the time-series of length 10:

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

If we want to produce a set of features using lag two intervals in the past then the first two element of the time-series are considered a burnt-in sample. We can't use the observations associated with them to train out algorithm.

The raw values, of 8 rows by 2 columns would be

[[ 1.,  0.]
 [ 2.,  1.],
 [ 3.,  2.],
 [ 4.,  3.],
 [ 5.,  4.],
 [ 6.,  5.],
 [ 7.,  6.],
 [ 8.,  7.]]

The differenced values

[[ 1.,  1.],
 [ 1.,  1.],
 [ 1.,  1.],
 [ 1.,  1.],
 [ 1.,  1.],
 [ 1.,  1.],
 [ 1.,  1.]])

These get column stacked. There are many additional features you could explore. Rolling mean would be my next pick.

If you want to predict further in the future then your training data should be lagging further from your label vector.

If performance isn't satisfactory then try adding more features by choosing a rolling mean over a bigger window, or add further back in the future. A clever trick to improve the performance of time-series algorithms is to include the value of the prediction for the previous time interval.

Fit your classifier on some early part of the data, then observe its accuracy over a later part of the data. There are many metrics for classifiers you can use. If you choose to use a classifier that outputs probabilities instead of hard 1/0, then your options even broaden. (As does the uses of your classifier.)

Precision and recall are intuitive performance metrics of classifiers.

Train on the first (early) half of your data and test on the second half (later).

As far as algorithms go, I'd look into logistic regression. I'd only look elsewhere if the performance isn't satisfactory and you've exhausted feature extraction options.

Mallet appears to be a good library for the task. See this bit of the docs.

I recently discovered JSAT, which looks promising.

There are more specific approaches to time-series classification that explicitly take into account the sequential nature of the observations and labels. This is a general purpose adaptation of classification to time-series.

Answer 4

If you are interested in using support vector machines, there's a guide that is very oriented for beginners and you might find useful (http://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf)

That guide is from the same guys of libsvm which is a very mature library for support vector machines (http://www.csie.ntu.edu.tw/~cjlin/libsvm/) and they do have binding for Java (http://www.csie.ntu.edu.tw/~cjlin/libsvm/#java)