Non-linear multivariate time-series response prediction using RNN

前端 未结 1 898
长发绾君心
长发绾君心 2021-01-31 23:51

I am trying to predict the hygrothermal response of a wall, given the interior and exterior climate. Based on literature research, I believe this should be possible with RNN but

1条回答
  •  南笙
    南笙 (楼主)
    2021-02-01 00:01

    In the end, I managed to solve this the following way:

    • Using more samples to train instead of only 1 (I used 18 samples to train and 6 to test)
    • Keep the first year of data, as the output time-series for all samples have the same 'starting point' and the model needs this information to learn
    • Standardise both input and output features (zero mean, unit variance). I found this improved prediction accuracy and training speed
    • Use stateful LSTM as described here, but add reset states after epoch (see below for code). I used batch_size = 6 and T_after_cut = 1460. If T_after_cut is longer, training is slower; if T_after_cut is shorter, accuracy decreases slightly. If more samples are available, I think using a larger batch_size will be faster.
    • use CuDNNLSTM instead of LSTM, this speed up the training time x4!
    • I found that more units resulted in higher accuracy and faster convergence (shorter training time). Also I found that the GRU is as accurate as the LSTM tough converged faster for the same number of units.
    • Monitor validation loss during training and use early stopping

    The LSTM model is build and trained as follows:

    def define_reset_states_batch(nb_cuts):
      class ResetStatesCallback(Callback):
        def __init__(self):
          self.counter = 0
    
        def on_batch_begin(self, batch, logs={}):
        # reset states when nb_cuts batches are completed
          if self.counter % nb_cuts == 0:
            self.model.reset_states()
          self.counter += 1
    
        def on_epoch_end(self, epoch, logs={}):
        # reset states after each epoch
          self.model.reset_states()
          return(ResetStatesCallback)    
    
    model = Sequential()
    model.add(layers.CuDNNLSTM(256, batch_input_shape=(batch_size,T_after_cut ,features),
      return_sequences=True,
      stateful=True))
    model.add(layers.TimeDistributed(layers.Dense(targets, activation='linear')))
    
    optimizer = RMSprop(lr=0.002)
    model.compile(loss='mean_squared_error', optimizer=optimizer)
    
    earlyStopping = EarlyStopping(monitor='val_loss', min_delta=0.005, patience=15, verbose=1, mode='auto')
    ResetStatesCallback = define_reset_states_batch(nb_cuts)
    model.fit(X_dev, y_dev, epochs=n_epochs, batch_size=n_batch, verbose=1, shuffle=False, validation_data=(X_eval,y_eval), callbacks=[ResetStatesCallback(), earlyStopping])
    

    This gave me very statisfying accuracy (R2 over 0.98): This figure shows the temperature (left) and relative humidity (right) in the wall over 2 years (data not used in training), prediction in red and true output in black. The residuals show that the error is very small and that the LSTM learns to capture the long-term dependencies to predict the relative humidity.

    0 讨论(0)
提交回复
热议问题