学习率与动量的关系：此涨彼消，以达到动态平衡

选择适当大小的学习率和动量，太大会使loss振荡，太小不足以使loss达到最优值。

1、train_gluon_ch7(‘sgd’, {‘learning_rate’: 0.004, ‘momentum’: 0.99}, features, labels)
loss: 0.301315, 0.176527 sec per epoch

2、train_gluon_ch7(‘sgd’, {‘learning_rate’: 0.004, ‘momentum’: 0.95}, features, labels)
loss: 0.256159, 0.194480 sec per epoch

3、train_gluon_ch7(‘sgd’, {‘learning_rate’: 0.004, ‘momentum’: 0.9}, features, labels)
loss: 0.243600, 0.181515 sec per epoch

4、train_gluon_ch7(‘sgd’, {‘learning_rate’: 0.004, ‘momentum’: 0.8}, features, labels)
loss: 0.244236, 0.172539 sec per epoch

5、train_gluon_ch7(‘sgd’, {‘learning_rate’: 0.004, ‘momentum’: 0.5}, features, labels)
loss: 0.255883, 0.189493 sec per epoch

6、train_gluon_ch7(‘sgd’, {‘learning_rate’: 0.02, ‘momentum’: 0.8}, features, labels)
loss: 0.243856, 0.181516 sec per epoch

7、train_gluon_ch7(‘sgd’, {‘learning_rate’: 0.02, ‘momentum’: 0.5}, features, labels)
loss: 0.244130, 0.182513 sec per epoch

8、train_gluon_ch7(‘sgd’, {‘learning_rate’: 0.02, ‘momentum’: 0.45}, features, labels)
loss: 0.243134, 0.210438 sec per epoch

9、train_gluon_ch7(‘sgd’, {‘learning_rate’: 0.02, ‘momentum’: 0.4}, features, labels)
loss: 0.245365, 0.199467 sec per epoch

10、train_gluon_ch7(‘sgd’, {‘learning_rate’: 0.02, ‘momentum’: 0.2}, features, labels)
loss: 0.244297, 0.187498 sec per epoch

11、train_gluon_ch7(‘sgd’, {‘learning_rate’: 0.02, ‘momentum’: 0.1}, features, labels)
loss: 0.243998, 0.174533 sec per epoch

来源：CSDN

作者：立昂

链接：https://blog.csdn.net/nielinyuan1466/article/details/103914321