recurrent-neural-network

问题 I want to train an RNN language model using TensorFlow. My training data is a sequence of 5 tokens represented with integers like so x = [0, 1, 2, 3, 4] I want the unrolled length of the RNN to be 4, and the training batch size to be 2. (I chose these values in order to require padding.) Each token has an embedding of length 3 like so 0 -> [0, 0 ,0] 1 -> [10, 10, 10] 2 -> [20, 20, 20] 3 -> [30, 30, 30] 4 -> [40, 40, 40] What should I pass as parameters to tf.nn.dynamic_rnn ? This is mostly a

问题 I'm having a tough time using the 'initial state' argument in the tf.nn.rnn function. val, _ = tf.nn.rnn(cell1, newBatch, initial_state=stateP, dtype=tf.float32) newBatch.shape => (1, 1, 11) stateP.shape => (2, 2, 1, 11) In general, I've gone through the training for my LSTM neural net and now I want to use the values of it. How do I do this? I know that the tf.nn.rnn() function will return state... but I don't know how to plug it in. fyi stateP.shape => (2, 2, 1, 11) ..... maybe because I

问题 I am attempting to build two similar models predicting different output types. One predicts between two categories and the other has six output categories. Their inputs are the same and they are both LSTM RNN. I have separated training and predicting out into separate functions in each of their files, model1.py, model2.py. I have made the mistake of naming variables in each model the same thing so that when I call predict1 and predict2 from model1 and model2 respectively I get the following

问题 I have run a Keras LSTM demo containing the following code (after line 166): m = 1 model=Sequential() dim_in = m dim_out = m nb_units = 10 model.add(LSTM(input_shape=(None, dim_in), return_sequences=True, units=nb_units)) model.add(TimeDistributed(Dense(activation='linear', units=dim_out))) model.compile(loss = 'mse', optimizer = 'rmsprop') When I prepend a call to model.summary() , I see the following output: _________________________________________________________________ Layer (type)

问题 I have a mechanical problem as kind of a time series with raw data as follows time dtime cur dcur type proc start end 122088 1554207711521 3140 0.766106 0.130276 0 87556 1554203520000 1554207720000 122089 1554207714411 1800 0.894529 0.089670 0 87556 1554203520000 1554207720000 For every proc , there is a time series with time-instances not exactly in proper intervals. I have data from a set of different proc s, each coming from the same type of mechanical problem. The target is to predict the

问题 Regarding the answer provided by @Shai in LSTM module for Caffe, where caffe.NetSpec() is used to explicitly unroll LSTM units in time for training. Using this code implementation, why does the "DummyData" layer, or any data layer used instead as input X , appears at the end of the t0 time step, just before "t1/lstm/Mx" in the prototxt file? I don't get it... A manipulation (cut / paste) is hence needed. 回答1: Shai's NetSpec implementation of LSTM unrolls the net in time. Hence for every time

问题 I'm getting an error but it's buried down in the TensorFlow library so I'm struggling to figure out what's wrong with my model. I'm trying to use an RNN with LSTM. My model looks like this: model = Sequential() model.add(LSTM(128, activation='relu', input_shape=1000, return_sequences=True)) model.add(Dropout(0.2)) model.add(LSTM(128, activation='relu')) model.add(Dropout(0.2)) model.add(Dense(32, activation='relu')) model.add(Dropout(0.2)) model.add(Dense(2, activation='softmax')) opt = tf

问题 I do understand conceptually what an LSTM or GRU should (thanks to this question What's the difference between "hidden" and "output" in PyTorch LSTM?) BUT when I inspect the output of the GRU h_n and output are NOT the same while they should be... (Pdb) rnn_output tensor([[[ 0.2663, 0.3429, -0.0415, ..., 0.1275, 0.0719, 0.1011], [-0.1272, 0.3096, -0.0403, ..., 0.0589, -0.0556, -0.3039], [ 0.1064, 0.2810, -0.1858, ..., 0.3308, 0.1150, -0.3348], ..., [-0.0929, 0.2826, -0.0554, ..., 0.0176, -0

问题 I was going through this tutorial. I have a question about the following class code: class RNN(nn.Module): def __init__(self, input_size, hidden_size, output_size): super(RNN, self).__init__() self.input_size = input_size self.hidden_size = hidden_size self.output_size = output_size self.i2h = nn.Linear(input_size + hidden_size, hidden_size) self.i2o = nn.Linear(input_size + hidden_size, output_size) self.softmax = nn.LogSoftmax() def forward(self, input, hidden): combined = torch.cat((input,

问题 LSTM is good for predicting what is going to happen after a sequence, but I assume that we have many sequences and that each sequence corresponds to a class label. How can we use LSTM to classify these sequences? 回答1: LSTM can be used for prediction as well as classification tasks. For classification, you can follow most commonly used architectures that I have described below. However, you can build your own model depending on your requirement. As the output of LSTM (Here I explain dynamic