recurrent-neural-network

问题 I want to build a model, that predicts next character based on the previous characters. I have spliced text into sequences of integers with length = 100(using dataset and dataloader). Dimensions of my input and target variables are: inputs dimension: (batch_size,sequence length). In my case (128,100) targets dimension: (batch_size,sequence length). In my case (128,100) After forward pass I get dimension of my predictions: (batch_size, sequence_length, vocabulary_size) which is in my case (128

问题 I want to build a model, that predicts next character based on the previous characters. I have spliced text into sequences of integers with length = 100(using dataset and dataloader). Dimensions of my input and target variables are: inputs dimension: (batch_size,sequence length). In my case (128,100) targets dimension: (batch_size,sequence length). In my case (128,100) After forward pass I get dimension of my predictions: (batch_size, sequence_length, vocabulary_size) which is in my case (128

问题 I try to create model with RNN network but I receive : Input 0 of layer lstm_9 is incompatible with the layer: expected ndim=3, found ndim=4. Full shape received: [None, 2, 4000, 256] error. INPUT train_data.shape() = (100,2,4000) train_labels.shape() =(100,) labels_values = 0 or 1 (two classes) MODEL input = Input(shape=(2,4000)) # shape from train_data embedded = Embedding(2, 256)(input) lstm = LSTM(1024, return_sequences=True)(embedded) # ERROR dense = Dense(2, activation='softmax')(lstm)

问题 I want to implement an RNN using Tensorflow1.13 on GPU. Following the official recommendation, I write the following code to get a stack of RNN cells lstm = [tk.layers.CuDNNLSTM(128) for _ in range(2)] cells = tk.layers.StackedRNNCells(lstm) However, I receive an error message: ValueError: ('All cells must have a state_size attribute. received cells:', [< tensorflow.python.keras.layers.cudnn_recurrent.CuDNNLSTM object at 0x13aa1c940>]) How can I correct it? 回答1: This may be a Tensorflow bug

问题 The following is how I understand the point of parameter sharing in RNNs: In regular feed-forward neural networks, every input unit is assigned an individual parameter, which means that the number of input units (features) corresponds to the number of parameters to learn. In processing e.g. image data, the number of input units is the same over all training examples (usually constant pixel size * pixel size * rgb frames). However, sequential input data like sentences can come in highly

问题 I am trying to generate prediction intervals for a simple RNN using dropout. I'm using the functional API with training=True to enable dropout during testing. To try different dropout levels, I defined a small function to edit the model configs: from keras.models import Model, Sequential def dropout_model(model, dropout): conf = model.get_config() for layer in conf['layers']: if layer["class_name"]=="Dropout": layer["config"]["rate"] = dropout elif "dropout" in layer["config"].keys(): layer[

问题 I know that applying a TimeDistributed(Dense) applies the same dense layer over all the timesteps but I wanted to know how to apply different dense layers for each timestep. The number of timesteps is not variable. P.S.: I have seen the following link and can't seem to find an answer 回答1: You can use a LocallyConnected layer. The LocallyConnected layer words as a Dense layer connected to each of kernel_size time_steps (1 in this case). from tensorflow import keras from tensorflow.keras.layers

问题 I know that applying a TimeDistributed(Dense) applies the same dense layer over all the timesteps but I wanted to know how to apply different dense layers for each timestep. The number of timesteps is not variable. P.S.: I have seen the following link and can't seem to find an answer 回答1: You can use a LocallyConnected layer. The LocallyConnected layer words as a Dense layer connected to each of kernel_size time_steps (1 in this case). from tensorflow import keras from tensorflow.keras.layers

问题 I ran into some memory issues (GPU) when running a large RNN network, but I want to keep my batch size reasonable so I wanted to try out gradient accumulation. In a network where you predict the output in one go, that seems self-evident but in an RNN you do multiple forward passes for each input step. Because of that, I fear that my implementation does not work as intended. I started from user albanD's excellent examples here , but I think they should be modified when using an RNN. The reason

问题 After running my model for one epoch it crashed with following error message: InvalidArgumentError: Specified a list with shape [60,9] from a tensor with shape [56,9] [[{{node TensorArrayUnstack/TensorListFromTensor}}]] [[sequential_7/lstm_17/PartitionedCall]] [Op:__inference_train_function_29986] This happened after I changed the LSTM Layer to stateful=True and had to pass the batch_input_shape Argument instead of the input_shape Below is my code, I'm sure it has something to do with the