TensorBoard: How to plot histogram for gradients?

Question

TensorBoard had the function to plot histograms of Tensors at session-time. I want a histogram for the gradients during training.

tf.gradients(yvars,xvars)

Answer 1

Another solution (based on this quora answer) is to access the gradients directly from the optimizer you are already using.

optimizer = tf.train.AdamOptimizer(..)
grads = optimizer.compute_gradients(loss)
grad_summ_op = tf.summary.merge([tf.summary.histogram("%s-grad" % g[1].name, g[0]) for g in grads])
grad_vals = sess.run(fetches=grad_summ_op, feed_dict = feed_dict)
writer['train'].add_summary(grad_vals)