Is tensorflow lazy?

Question

Let\'s say you have some piece of code like this

import tensorflow as tf
...
f = h*y + z*t  #Just some expression involving other tensors.
e = ... # some exp         


        

Answer 1

TL;DR: TensorFlow is strict, so both e and f will be evaluated before the tf.select() node executes.

This has caused some confusion. TensorFlow first prunes the dataflow graph based on which operations are statically required to produce the values that are fetched (i.e. the arguments to sess.run()). Once the graph has been pruned, however, the runtime uses strict execution, whereby all of the inputs to an operation (such as tf.select()) must have been computed before that operation can execute.

There is experimental support for conditional execution in the tf.control_flow_ops module, using the tf.control_flow_ops.cond() function, but this is sparsely documented at present.