How can I apply multithreading to the backpropagation neural network training?

Question

For my university project I am creating a neural network that can classify the likelihood that a credit card transaction is fraudulent or not. I am training with backpropag

Answer 1

For using more CPU cores with Java you can just try more options for JVM without changing the code:

• -server
• -d64
• -XX:-UseParallelGC

and other options at http://java.sun.com/javase/technologies/hotspot/vmoptions.jsp

Answer 2

First of all don't use backpropagation. There are many other options out there. I would suggest trying RPROP (resilient propagation). It won't be that big of modification to your backpropagation algorithm. You do not need to specify learning rate or momentum. Its really almost as if you have an individual, variable, learning rate for every connection in the neural network.

As to applying multithreading to backpropagation. I just wrote an article on this topic.

http://www.heatonresearch.com/encog/mprop/compare.html

Basically I create a number of threads and divide up the training data so each thread has a near equal amount. I am calculating the gradients in each thread and they are summed in a reduce step. How the gradients are applied to the weights depends on the propagation training algorithm used, but the weight update is done in a critical section.

When you have considerably more training samples than weights the code spends much more time in the multi-threaded gradient calculation than the critical section weight update.

I provide some of the performance results at the above link. It does really speed things up!