Where can I have a look at TensorFlow gradient descent main loop?

Question

(Sorry if this sounds a bit naive) I want to have a look at the meat of the TensorFlow implementation for GradientDescent - and see for myself how are they handling termination

Answer 1

TensorFlow Optimizer interface, (which GradientDescentOptimizer implements) defines a a single step of minimization. Termination conditions or adjusting step size is implemented by the user. In MNIST for Beginners tutorial, the termination conditions is "stop after 1000" steps which you can see in for i in range(1000) loop

apply_gradient_descent(a,b,c) is a fused op that multiplies c by b and adds it to a. There are some extra levels of indirection to go from Python wrapper to C++ implementation detailed in Adding a new op HowTo, but as a shortcut you can usually find C++ implementation by converting from snake-case and searching for that, so ApplyGradientDescent in this case. That leads to implementation in tensorflow/core/kernels/training_ops.cc