Intuition for perceptron weight update rule

前端 未结 2 794
难免孤独
难免孤独 2021-02-13 14:33

I am having trouble understanding the weight update rule for perceptrons:

w(t + 1) = w(t) + y(t)x(t).

Assume we have a linearly separable data set.

2条回答
  •  广开言路
    2021-02-13 14:49

    The perceptron's output is the hard limit of the dot product between the instance and the weight. Let's see how this changes after the update. Since

    w(t + 1) = w(t) + y(t)x(t),

    then

    x(t) ⋅ w(t + 1) = x(t) ⋅ w(t) + x(t) ⋅ (y(t) x(t)) = x(t) ⋅ w(t) + y(t) [x(t) ⋅ x(t))].


    Note that:

    • By the algorithm's specification, the update is only applied if x(t) was misclassified.
    • By the definition of the dot product, x(t) ⋅ x(t) ≥ 0.

    How does this move the boundary relative to x(t)?

    • If x(t) was correctly classified, then the algorithm does not apply the update rule, so nothing changes.
    • If x(t) was incorrectly classified as negative, then y(t) = 1. It follows that the new dot product increased by x(t) ⋅ x(t) (which is positive). The boundary moved in the right direction as far as x(t) is concerned, therefore.
    • Conversely, if x(t) was incorrectly classified as positive, then y(t) = -1. It follows that the new dot product decreased by x(t) ⋅ x(t) (which is positive). The boundary moved in the right direction as far as x(t) is concerned, therefore.

提交回复
热议问题