Sklearn Implementation for batch gradient descend

别等时光非礼了梦想. 提交于 2020-06-09 06:10:07

问题


What is the way of implementing Batch gradient descent using sklearn for classification? We have SGDClassifier for Stochastic GD which will take single instance at a time and Linear/Logistic Regression which uses normal equation.


回答1:


The possible answer to the question as pointed out in the other similar question as well from sklearn docs:

SGD allows minibatch (online/out-of-core) learning, see the partial_fit method.

But is partial_fit really a batch gradient decent ?

SGD: The gradient of the cost function is calculated and the weights are updated using the gradient decent step for each sample.

Batch/Mini Batch SGD: The gradient of the cost function is calculated and the weights are updated using the gradient decent step once per batch.

So Batch SGD with batch size of 1 == SGD

Now that we are clear about definitions lets investigate the code of sklearn SGDClassifier

The docstring of partial_fit says

Perform one epoch of stochastic gradient descent on given samples.

But this is not a batch SGD but its looks more like a helper function to run fit method with max_iter=1 (infact commented as same in docstrings).

partial_fit calls _partial_fit with max_iter==1 Reference link

fit method calls _fit which calls _partial_fit with max_iter set to the assigned\default maximum iterations. Reference link

conclusion:

partial_fit does not really do batch SGD, i.e it is not calculating the gradients and updating the weight per batch but rather doing so for each sample.

There seems to be no mechanism in sklearn to do batch gradient descend.



来源:https://stackoverflow.com/questions/55552139/sklearn-implementation-for-batch-gradient-descend

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!