问题
I am training a deep residual network with 10 hidden layers with game data.
Does anyone have an idea why I don't get any overfitting here? Training and test loss still decreasing after 100 epochs of training.
https://imgur.com/Tf3DIZL
回答1:
Just a couple of advice:
- for deep learning is recommended to do even 90/10 or 95/5 splitting (Andrew Ng)
- this small difference between curves means that your
learning_rate
is not tuned; try to increase it (and, probably, number ofepochs
if you will implement some kind of 'smart' lr-reduce) - it is also reasonable for DNN to try to overfit with the small amount of data (10-100 rows) and an enormous number of iterations
- check for data leakage in the set: weights analysis inside each layer may help you in this
来源:https://stackoverflow.com/questions/54489549/why-does-my-neural-network-never-overfit