What is the difference between backpropagation and reverse-mode autodiff?

后端 未结 3 678
无人共我
无人共我 2021-02-13 23:20

Going through this book, I am familiar with the following:

For each training instance the backpropagation algorithm first makes a prediction (forward pa

3条回答
  •  情书的邮戳
    2021-02-13 23:47

    Thanks to the answer by David Parks for the valid contribution and useful links, however I have found the answer to this question by the author of the book himself, which may provide a more concise answer:

    Bakpropagation refers to the whole process of training an artificial neural network using multiple backpropagation steps, each of which computes gradients and uses them to perform a Gradient Descent step. In contrast, reverse-mode auto diff is simply a technique used to compute gradients efficiently and it happens to be used by backpropagation.

提交回复
热议问题