Training an LSTM neural network to forecast time series in pybrain, python

Question

I have a neural network created using PyBrain and designed to forecast time series.

I am using the sequential dataset function, and trying to use a sliding window of 5 previous values to predict the 6th. One of my problems is that I can't figure out how to create the required dataset by appending the 5 previous values to the inputs and the 6th as an output.

I am also unsure of how exactly to forecast values in the series once the network is trained.

Posting my code below:

from pybrain.datasets import SupervisedDataSet
from pybrain.datasets import SequentialDataSet
from pybrain.tools.shortcuts import buildNetwork
from pybrain.supervised.trainers import BackpropTrainer
from pybrain.supervised.trainers import RPropMinusTrainer
from pylab import ion, ioff, figure, draw, contourf, clf, show, hold, plot
from pybrain.structure import RecurrentNetwork
from pybrain.structure import FeedForwardNetwork
from pybrain.structure import LinearLayer, SigmoidLayer, TanhLayer
from pybrain.structure import FullConnection
from pybrain.structure import LSTMLayer
from pybrain.structure import BiasUnit
from pybrain.rl.learners.valuebased import Q
import pybrain
import matplotlib as plt
import translate
import time
import pickle
import scipy as sp
import numpy as np
import pylab as pl
import itertools

#Opening data from database
data = translate.translate(3600)
time, price, volume = zip(*data)

#Creating data lists instead of tuples
timeList = []
priceList = []
volumeList = []

for record in time:
    timeList.append(record)

for record in price:
    priceList.append(record)

for record in volume:
    volumeList.append(record)

#Creating lookback window and target
datain = priceList[:5] 
dataout = priceList[6]
print datain
print dataout
#Creating the dataset
ds = SequentialDataSet(5, 1)

for x, y in itertools.izip(datain, dataout):
    ds.newSequence()
    ds.appendLinked(tuple(x), tuple(y))
    print (x, y)

print ds

#Building the network
n = RecurrentNetwork()

#Create the network modules
n.addInputModule(SigmoidLayer(5, name = 'in'))
n.addModule(LSTMLayer(100, name = 'LSTM'))
n.addModule(LSTMLayer(100, name = 'LSTM2'))
n.addOutputModule(SigmoidLayer(1, name = 'out'))

#Add the network connections
n.addConnection(FullConnection(n['in'], n['LSTM'], name = 'c_in_to_LSTM'))
n.addConnection(FullConnection(n['in'], n['LSTM2'], name = 'c_in_to_LSTM2'))
n.addConnection(FullConnection(n['LSTM'], n['out'], name = 'c_LSTM_to_out'))
n.addConnection(FullConnection(n['LSTM2'], n['out'], name = 'c_LSTM2_to_out'))

n.sortModules()
n.randomize()

#Creating the trainer
trainer = BackpropTrainer(n, ds)

#Training the network
#for i in range (1000):
#   print trainer.train()

#Make predictions

#Plotting the results
pl.plot(time, price)


pl.show()

The above code gives: TypeError: izip argument #2 must support iteration

I have seen the question linked below however I haven't been successful

Event Sequences, Recurrent Neural Networks, PyBrain

First question on this great site, any help is appreciated

Answer 1

#Creating lookback window and target datain = priceList[:5] dataout = priceList[6]

Not an expert. But it seems your datain is a list with length=6 while dataout is not.

Answer 2

I'd guess the TypeError says it all. Whereas priceList[:5] is a list and hence iterable, priceList[6] is a single element.

You'd probably want something like

datain = priceList[:5] 
dataout = priceList[6:6]

which will make dataout a list with a single element.