Purpose of decay parameter in nnet function in R?

后端 未结 2 646
你的背包
你的背包 2021-02-08 14:50

I am using nnet function in R to train my neural network. I am not getting what is decay parameter in nnet is? Is this step size to be used in gradient descent mentod or regular

2条回答
  •  忘了有多久
    2021-02-08 15:15

    It's regularization to avoid over-fitting.

    From the documentation (pdf):

    decay: parameter for weight decay. Default 0.

    Further information is available in the authors' book, Modern Applied Statistics with S. Fourth Edition, page 245:

    One way to ensure that f is smooth is to restrict the class of estimates, for example, by using a limited number of spline knots. Another way is regularization in which the fit criterion is altered to

    E + λC(f)

    with a penalty C on the ‘roughness’ of f . Weight decay, specific to neural networks, uses as penalty the sum of squares of the weights wij. ... The use of weight decay seems both to help the optimization process and to avoid over-fitting. (emphasis added)

提交回复
热议问题