temporal-difference

问题 I have a read few papers and lectures on temporal difference learning (some as they pertain to neural nets, such as the Sutton tutorial on TD-Gammon) but I am having a difficult time understanding the equations, which leads me to my questions. -Where does the prediction value V_t come from? And subsequently, how do we get V_(t+1)? -What exactly is getting back propagated when TD is used with a neural net? That is, where does the error that gets back propagated come from when using TD? 回答1:

问题 I am attempting to construct a neural network incorporating convolution and LSTM (using the Torch library) to be trained by Q-learning or Advantage-learning, both of which require propagating state T+1 through the network before updating the weights for state T. Having to do an extra propagation would cut performance and that's bad, but not too bad; However, the problem is that there is all kinds of state bound up in this. First of all, the Torch implementation of backpropagation has some

问题 This is a small portion of the dataframe I am working with for reference.I am working with a data frame (MG53_HanLab) in R that has a column for Time, several columns with the name "MG53" in them, several columns with the name "F2" and several with "Iono" in them. I would like to compare the means of each group for each time point. I understand that I have to subset the data and have tried doing control <- MG53_HanLab[c(2:11)] F2 <- MG53_HanLab[c(12:23)] iono <- MG53_HanLab[c(24:33)] which