Tensorflow execution time

Question

I have a function within a Python script that I am calling multiple times (https://github.com/sankhaMukherjee/NNoptExpt/blob/dev/src/lib/NNlib/NNmodel.py): I have simplified

Answer 1

You are calling tf.assign in the the session context. This will keep adding ops to your graph every time you execute the errorValW function, slowing down execution as your graph grows larger. As a rule of thumb, you should avoid ever calling Tensorflow ops when executing models on data (since this will usually be inside a loop, resulting in constant growth of the graph). From my personal experience, even if you are only adding "a few" ops during execution time this can result in extreme slowdown.

Note that tf.assign is an op like any other. You should define it once beforehand (when creating the model/building the graph) and then run the same op repeatedly after launching the session.

I don't know what exactly you are trying to achieve in your code snippet, but consider the following:

...
with tf.Session() as sess:
    sess.run(tf.assign(some_var, a_value))

could be replaced by

a_placeholder = tf.placeholder(type_for_a_value, shape_for_a_value)
assign_op = tf.assign(some_var, a_placeholder)
...
with tf.Session() as sess:
    sess.run(assign_op, feed_dict={a_placeholder: a_value})

where a_placeholder should have the same dtype/shape as some_var. I have to admit I haven't tested this snippet so please let me know if there are issues, but this should be about right.