Tensorflow provides all sorts of nice streaming operations to aggregate statistics along batches, such as tf.metrics.mean.
tf.metrics.mean
However I find that accumulating
The metrics in tf.contrib.eager.metrics (which work both with and without eager execution) have a init_variable() op you can call if you want to reset their internal variables.
tf.contrib.eager.metrics
init_variable()