Simple gradient descent using mxnet

流过昼夜 提交于 2019-12-05 15:59:44

Currently it is not as easy as tensorflow to optimize a simple function using MXNet, due to the lack of support in frontend.

First you need a loss Function as the last layer of your network. Here it is log_x_squared. Use MakeLoss to create a loss function.

Second is the input and weights. Since currently in MXNet Variable is not counted as trainable weight, you need to set x as weight. Here is a workaround: Set a 'fake' input variable which is always to be 1. After it add a fullyconnected layer with 1 hidden unit and no bias. This gives us "1 * x". Now our x is a weight.

Third if you would like to optimize multiple times on single data sample, module.fit might not be the best choice. After initializing optimizer. You just need to call module.forward_backward() and module.update() multiple times. For forward_backward function you need to pass a databatch, which is a simpler interface comparing to dataiter. Here we just need to pass a constant ndarray of 1 every time.

Actually we construct a computation graph of log(1 * x) ^ 2 and x becomes a weight instead of variable.

Anyway, we should consider providing a similar interface of tensorflow to optimize variable.

Hope this is useful info!

mxnet is different from tensorflow. You need to read this tutorial

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!