How to release tensors from memory when not in need anymore?

Question

I have hypothetical graph which has a series of computations as follows:

a_0 = tf.placeholder()
a_1 = some_op_1(a_0)
a_2 = some_op_2(a_1)
a_3 = some_op_3(a_2         


        

Answer 1

TensorFlow uses reference counting to release the memory used by a tensor as soon as it is no longer used. The values of a_0 and a_1 will be deleted as soon as there are no more references to them, and in the latest builds of TensorFlow (post-1.0 nightly builds) some operations will even reuse the input buffer for the output if they have the same shape and element type.