recurrent-neural-network

问题 Goal I have a strange situation trying to create an efficient autoencoder over my time series dataset: X_train (200, 23, 178) X_val (100, 23, 178) X_test (100, 23, 178) Current situation With a simple autoencoder I have better results rather than my simple LSTM AE over a dataset of time series. I have some concerns about my utilization of the Repeat Vector wrapper layer, which as far as I understood, is supposed to repeat a number of times like the sequence length the last state of the LSTM

问题 I have two question on Tensorflow PTB RNN tutorial code ptb_word_lm.py. Code blocks below are from the code. Is it okay to reset state for every batch? self._initial_state = cell.zero_state(batch_size, data_type()) with tf.device("/cpu:0"): embedding = tf.get_variable( "embedding", [vocab_size, size], dtype=data_type()) inputs = tf.nn.embedding_lookup(embedding, input_.input_data) if is_training and config.keep_prob < 1: inputs = tf.nn.dropout(inputs, config.keep_prob) outputs = [] state =

问题 I'm trying to understand RNNs (not a specific one) with the Reber Grammar inputs (not embedded for now). You can find the jupyter notebook on this link (please disregard markdowns because I failed on the first version with output and it's not up-to-date :) ) . For every timestep, I provide the input and expected output for the training (so it's a many-to-many model). Input/output are "OneHotEncoded" (based on the string "BTSXPVE") so for example B is [1, 0, 0, 0, 0, 0, 0] V is [0, 0, 0, 0, 0,

问题 I'm trying to understand RNNs (not a specific one) with the Reber Grammar inputs (not embedded for now). You can find the jupyter notebook on this link (please disregard markdowns because I failed on the first version with output and it's not up-to-date :) ) . For every timestep, I provide the input and expected output for the training (so it's a many-to-many model). Input/output are "OneHotEncoded" (based on the string "BTSXPVE") so for example B is [1, 0, 0, 0, 0, 0, 0] V is [0, 0, 0, 0, 0,

问题 I have some very basic lstm code with tensorflow and python, where my code is output = tf.nn.rnn(tf.nn.rnn_cell.BasicLSTMCell(10), input_flattened, initial_state=tf.placeholder("float", [None, 20])) where my input flattened is shape [?, 5, 22501] I'm getting the error TypeError: inputs must be a sequence on the state parameter of the lstm, and I'm ripping my hair out trying to find out why it is giving me this error. Any help would be greatly appreciated. 回答1: I think when you use the tf.nn

问题 I was training Bidirectional LSTM type RNN for nearly 24 hours, and due to oscillation in the error I decided to decrease the learning before allowing it to continue training. Since the model is saved using Saver.save(sess,file) at every epoch, I terminated the training with the CTC Loss having minimised to approximately 115. Now after restoring the model, the initial error rate I am getting is somewhere around 162, which is inconsistent with the flow of error rate I was getting in 7th epoch,

问题 I use the Elman recurrent network from neurolab to predict a time series of continuous values. The network is trained from a sequence such that the input is the value at index i and the target is the value at index i+1 . To make predictions beyond the immediate next time step, the output of the net is feed back as input. If, for example, I intend to predict the value at i+5 , I proceed as follows. Input the value from i Take the output and feed it to the net the as next input value (e.g. i+1

问题 Below I have an implementation of a Tensorflow RNN Cell, designed to emulate Alex Graves' algorithm ACT in this paper: http://arxiv.org/abs/1603.08983. At a single timestep in the sequence called via rnn.rnn(with a static sequence_length parameter, so the rnn is unrolled dynamically - I am using a fixed batch size of 20), we recursively call ACTStep, producing outputs of size(1,200) where the hidden dimension of the RNN cell is 200 and we have a batch size of 1. Using the while loop in

问题 Has anyone been able to mix feedforward layers and recurrent layers in Tensorflow? For example: input->conv->GRU->linear->output I can imagine one can define his own cell with feedforward layers and no state which can then be stacked using the MultiRNNCell function, something like: cell = tf.nn.rnn_cell.MultiRNNCell([conv_cell,GRU_cell,linear_cell]) This would make life a whole lot easier... 回答1: can't you just do the following: rnnouts, _ = rnn(grucell, inputs) linearout = [tf.matmul(rnnout,

问题 I have a audio data set and each of them has different length. There are some events in these audios, that I want to train and test but these events are placed randomly, plus the lengths are different, it is really hard to build a machine learning system with using that dataset. I thought fixing a default size of length and build a multilayer NN however, the length's of events are also different. Then I thought about using CNN, like it is used to recognise patterns or multiple humans on an