Why does my neural network never overfit?

眉间皱痕 提交于 2019-12-13 09:42:37

问题


I am training a deep residual network with 10 hidden layers with game data.

Does anyone have an idea why I don't get any overfitting here? Training and test loss still decreasing after 100 epochs of training.

https://imgur.com/Tf3DIZL


回答1:


Just a couple of advice:

  1. for deep learning is recommended to do even 90/10 or 95/5 splitting (Andrew Ng)
  2. this small difference between curves means that your learning_rate is not tuned; try to increase it (and, probably, number of epochs if you will implement some kind of 'smart' lr-reduce)
  3. it is also reasonable for DNN to try to overfit with the small amount of data (10-100 rows) and an enormous number of iterations
  4. check for data leakage in the set: weights analysis inside each layer may help you in this


来源:https://stackoverflow.com/questions/54489549/why-does-my-neural-network-never-overfit

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!