recurrent-neural-network

问题 I am trying to predict the hygrothermal response of a wall, given the interior and exterior climate. Based on literature research, I believe this should be possible with RNN but I have not been able to get good accuracy. The dataset has 12 input features (time-series of exterior and interior climate data) and 10 output features (time-series of hygrothermal response), both containing hourly values for 10 years. This data was created with hygrothermal simulation software, there is no missing

问题 I am new to LSTMs and going through the Understanding Keras LSTMs and had some silly doubts related to a beautiful answer by Daniel Moller. Here are some of my doubts: There are 2 ways specified under the Achieving one to many section where it’s written that we can use stateful=True to recurrently take the output of one step and serve it as the input of the next step (needs output_features == input_features). In the One to many with repeat vector diagram, the repeated vector is fed as input

问题 In this blog post, The Unreasonable Effectiveness of Recurrent Neural Networks, Andrej Karpathy mentions future directions for neural networks based machine learning: The concept of attention is the most interesting recent architectural innovation in neural networks. [...] soft attention scheme for memory addressing is convenient because it keeps the model fully-differentiable, but unfortunately one sacrifices efficiency because everything that can be attended to is attended to (but softly).

问题 Sources There are several sources out there explaining stateful / stateless LSTMs and the role of batch_size which I've read already. I'll refer to them later in my post: [1] https://machinelearningmastery.com/understanding-stateful-lstm-recurrent-neural-networks-python-keras/ [2] https://machinelearningmastery.com/stateful-stateless-lstm-time-series-forecasting-python/ [3] http://philipperemy.github.io/keras-stateful-lstm/ [4] https://machinelearningmastery.com/use-different-batch-sizes

问题 I'm trying to make RNN using LSTM . I made LSTM model, and after it, there is two DNN network, and one regression output layer. I trained my data, and the final training loss become about 0.009 . However, when i applied the model to test data, the loss become about 0.5 . The 1th epoch training loss is about 0.5 . So, I think the trained variable do not used in test model. The only difference between training and test model is batch size. Trainning Batch = 100~200 , Test Batch Size = 1 . in

问题 Environment info Operating System: Windows 7 64-bit Tensorflow installed from pre-built pip (no CUDA): 1.0.1 Python 3.5.2 64-bit Problem I have problems with restoring my net (RNN character base language model). Below is a simplified version with the same problem. When I run it the first time, I get, for example, this. ... step 160: loss = 1.956 (perplexity = 7.069016620211226) step 180: loss = 1.837 (perplexity = 6.274748642468816) step 200: loss = 1.825 (perplexity = 6.202084762557817) But

问题 Environment info Operating System: Windows 7 64-bit Tensorflow installed from pre-built pip (no CUDA): 1.0.1 Python 3.5.2 64-bit Problem I have problems with restoring my net (RNN character base language model). Below is a simplified version with the same problem. When I run it the first time, I get, for example, this. ... step 160: loss = 1.956 (perplexity = 7.069016620211226) step 180: loss = 1.837 (perplexity = 6.274748642468816) step 200: loss = 1.825 (perplexity = 6.202084762557817) But

问题 I'm trying to fit an RNN in Keras using sequences that have varying time lengths. My data is in a Numpy array with format (sample, time, feature) = (20631, max_time, 24) where max_time is determined at run-time as the number of time steps available for the sample with the most time stamps. I've padded the beginning of each time series with 0 , except for the longest one, obviously. I've initially defined my model like so... model = Sequential() model.add(Masking(mask_value=0., input_shape=

问题 I'm using bidirectional_rnn with GRUCell but this is a general question regarding the RNN in Tensorflow. I couldn't find how to initialize the weight matrices (input to hidden, hidden to hidden). Are they initialized randomly? to zeros? are they initialized differently for each LSTM I create? EDIT: Another motivation for this question is in pre-training some LSTMs and using their weights in a subsequent model. I don't currently know how to do that currently without saving all the states and

问题 EDITED for conciseness. I am trying to build an LSTM model, working off the documentation example at https://keras.io/layers/recurrent/ from keras.models import Sequential from keras.layers import LSTM The following three lines of code (plus comment) are taken directly from the documentation link above: model = Sequential() model.add(LSTM(32, input_dim=64, input_length=10)) # for subsequent layers, not need to specify the input size: model.add(LSTM(16)) ValueError: Input 0 is incompatible