Backpropagation through time

Question

Does anyone know of a library with a working implementation of backpropagation through time? Any of Java/Python/C#/VB.NET/F# (preferably the last one) will do!

Answer 1

Perhaps pybrain would do? The docstring for its BackpropTrainer class suggests that it does backpropagation through time:

class BackpropTrainer(Trainer):
    """Trainer that trains the parameters of a module according to a
    supervised dataset (potentially sequential) by backpropagating the errors
    (through time)."""

Answer 2

Assuming you're already using some library for BP, it should be (TM) rather straightforward to implement BPTT using BP as a step in the process.

The Wikipedia entry for BPTT [1] includes relevant pseudo code.

My own starting point, about 18 years ago, was "The Truck Backer-Upper: An Example of Self-Learning in Neural Networks" [2].

[1] http://en.wikipedia.org/wiki/Backpropagation_through_time

[2] http://www-isl.stanford.edu/~widrow/papers/c1989thetruck.pdf

Answer 3

I've used NeuronDotNet only for a limited time though. It allows you to create a feed-forward BackPropagation NN. I especially liked their use of intuitively named classes. Good luck!

This is a .net library.

Answer 4

You can use TensorFlow's dynamic_rnn() function (API doc). TensorFlow's tutorial on Recurrent Neural Networks will help.

Also, this great blog post provides a nice introduction to predicting sequences using TensorFlow. Here's another blog post with some code to predict a time series.

Answer 5

What about this one ? Just a Google search to help...

Answer 6

I've had good experiences with Weka - In my view one of the best and almost certainly the most comprehensive general purpose machine learning libraries around.

You could certainly do BPTT with Weka - you may find a ready made classifier that does what you need but even if not you can just chain a few normal backpropagation units together as per the very good wikipedia article on BPTT