tf.assign to variable slice doesn't work inside tf.while_loop

Question

What is wrong with the following code? The tf.assign op works just fine when applied to a slice of a tf.Variable if it happens outside of a loop.

Answer 1

I was executing this a few times and it isn't consistent. But variable slices do work inside the while loop.

Tried to split the graph inside body because the results are incorrect sometimes.

The correct answer (11, array([ 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89])) is returned sometimes but not always.

import tensorflow as tf

v = [1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0]
n = len(v)
a1 = tf.Variable(v, name = 'a')

def cond(i, _):
    return i < n

s = tf.InteractiveSession()
s.run(tf.global_variables_initializer())

def body( i, _):
    x = a1[i-1]
    y = a1[i-2]
    z = tf.add(x,y)
    op = a1[i].assign( z )
    with tf.control_dependencies([op]): #Edit This fixed the inconsistency.
       increment = tf.add(i, 1)
    return increment, op

print(s.run(tf.while_loop(cond, body, [2, a1])))

Answer 2

It makes sense from a CUDA perspective to disallow assignment of individual indices as it negates all performance benefits of heterogeneous parallel computing.

I know this adds a bit of computational overhead but it works.

import tensorflow as tf

v = [1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0]
n = len(v)
a = tf.Variable(v, name = 'a',dtype=tf.float32)

def cond(i, a):
    return i < n 

def body(i, a1):
    e = tf.eye(n,n)[i]
    a1 = a1 + e *(a1[i-1] + a1[i-2])
    return i + 1, a1

i, b = tf.while_loop(cond, body, [2, a]) 

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    print('i: ',sess.run(i))
    print('b: ',sess.run(b))

Answer 3

Your variable is not an output of the operations run inside your loop, it is an external entity living outside the loop. So you do not have to provide it as an argument.

Also, you need to enforce the update to take place, for example using tf.control_dependencies in body.

import tensorflow as tf

v = [1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0]
n = len(v)
a = tf.Variable(v, name = 'a')

def cond(i):
    return i < n 

def body(i):
    op = tf.assign(a[i], a[i-1] + a[i-2])
    with tf.control_dependencies([op]):
      return i + 1

i = tf.while_loop(cond, body, [2])

sess = tf.InteractiveSession()
tf.global_variables_initializer().run()
i.eval()
print(a.eval())
# [ 1  1  2  3  5  8 13 21 34 55 89]

Possibly you may want to be cautious and set parallel_iterations=1 to enforce the loop to run sequentially.