recurrent-neural-network

问题 I am building an RNN for classification (there is a softmax layer after the RNN). There are so many options for what to regularize and I am not sure if to just try all of them, would the effect be the same? which components do I regularize for what situation? The components being: Kernel weights (layer input) Recurrent weights Bias Activation function (layer output) 回答1: Regularizers that'll work best will depend on your specific architecture, data, and problem; as usual, there isn't a single

问题 I am trying to do some vanilla pattern recognition with an LSTM using Keras to predict the next element in a sequence. My data look like this: where the label of the training sequence is the last element in the list: X_train['Sequence'][n][-1] . Because my Sequence column can have a variable number of elements in the sequence, I believe an RNN to be the best model to use. Below is my attempt to build an LSTM in Keras: # Build the model # A few arbitrary constants... max_features = 20000 out

问题 I'm building a RNN loosely based on the TensorFlow tutorial. The relevant parts of my model are as follows: input_sequence = tf.placeholder(tf.float32, [BATCH_SIZE, TIME_STEPS, PIXEL_COUNT + AUX_INPUTS]) output_actual = tf.placeholder(tf.float32, [BATCH_SIZE, OUTPUT_SIZE]) lstm_cell = tf.nn.rnn_cell.BasicLSTMCell(CELL_SIZE, state_is_tuple=False) stacked_lstm = tf.nn.rnn_cell.MultiRNNCell([lstm_cell] * CELL_LAYERS, state_is_tuple=False) initial_state = state = stacked_lstm.zero_state(BATCH

问题 I'm building a RNN loosely based on the TensorFlow tutorial. The relevant parts of my model are as follows: input_sequence = tf.placeholder(tf.float32, [BATCH_SIZE, TIME_STEPS, PIXEL_COUNT + AUX_INPUTS]) output_actual = tf.placeholder(tf.float32, [BATCH_SIZE, OUTPUT_SIZE]) lstm_cell = tf.nn.rnn_cell.BasicLSTMCell(CELL_SIZE, state_is_tuple=False) stacked_lstm = tf.nn.rnn_cell.MultiRNNCell([lstm_cell] * CELL_LAYERS, state_is_tuple=False) initial_state = state = stacked_lstm.zero_state(BATCH

问题 I'm going through this tutorial on RNNs/LSTMs and I'm having quite a hard time understanding stateful LSTMs. My questions are as follows : 1. Training batching size In the Keras docs on RNNs, I found out that the hidden state of the sample in i -th position within the batch will be fed as input hidden state for the sample in i -th position in the next batch. Does that mean that if we want to pass the hidden state from sample to sample we have to use batches of size 1 and therefore perform

问题 I'm building a model that converts a string to another string using recurrent layers (GRUs). I have tried both a Dense and a TimeDistributed(Dense) layer as the last-but-one layer, but I don't understand the difference between the two when using return_sequences=True, especially as they seem to have the same number of parameters. My simplified model is the following: InputSize = 15 MaxLen = 64 HiddenSize = 16 inputs = keras.layers.Input(shape=(MaxLen, InputSize)) x = keras.layers.recurrent

问题 In my text generation dataset, I have converted all infrequent words into the token (unknown word), as suggested by most text-generation literature. However, when training an RNN to take in part of a sentence as input and predict the rest of the sentence, I am not sure how I should stop the network from generating tokens. When the network encounters an unknown (infrequent) word in the training set, what should its output be? Example: Sentence: I went to the mall and bought a <ukn> and some

问题 I have a stateful LSTM defined as a Sequential model: model = Sequential() model.add(LSTM(..., stateful=True)) ... Later, I use it as a Functional model: input_1, input_2 = Input(...), Input(...) output_1 = model(input_1) output_2 = model(input_2) # Is the state from input_1 preserved? Is the state from input_1 preserved when we apply model again on input_2 ? If yes, how can I reset the model state in between the calls? 回答1: Following Note on using statefulness in RNNs from this link and

问题 Here is my problem, I want to use one of the pretrain CNN network in a TimeDistributed layer. But I have some problem to implement it. Here is my model: def bnn_model(max_len): # sequence length and resnet input size x = Input(shape=(maxlen, 224, 224, 3)) base_model = ResNet50.ResNet50(weights='imagenet', include_top=False) for layer in base_model.layers: layer.trainable = False som = TimeDistributed(base_model)(x) #the ouput of the model is [1, 1, 2048], need to squeeze som = Lambda(lambda x

问题 This question already has answers here : Many to one and many to many LSTM examples in Keras (2 answers) Closed last year . I am reading this article (The Unreasonable Effectiveness of Recurrent Neural Networks) and want to understand how to express one-to-one, one-to-many, many-to-one, and many-to-many LSTM neural networks in Keras. I have read a lot about RNN and understand how LSTM NNs work, in particular vanishing gradient, LSTM cells, their outputs and states, sequence output and etc.