Tensorflow: what is the difference between tf.identity and '=' operator

问题

I'm confused about '=' operator, and tf.identity(), I thought '=' is to just make a reference of the tensor, and identity is to make a copy, e.g., with

ref = x
ref = ref*0
sess.run(x)

I will get x all been set to 0 element-wise, and with

copy = tf.identity(x)
copy = ref*0
sess.run(x)

x would not be changed, since identity make copy, not a reference, but with experiment, '=' also make a copy and x is not set to 0, so what's the difference?

回答1:

The difference is only in tensorlfow graph layout. tf.identity creates a new op in the graph that mimics its argument, while pure assignment adds a new python variable that points to the same op.

In both cases (the pair ref and x or the pair copy and x), both ops always evaluate to the same value. But in the second case, tf.get_default_graph().get_operations() will reveal a new operation in the list called Identity.

sess = tf.InteractiveSession()
x = tf.Variable(1.0)
sess.run(x.initializer)

ref = x
copy = tf.identity(x)
print(x.eval(), copy.eval(), ref.eval())  # 1.0 1.0 1.0

sess.run(x.assign(2.0))
print(x.eval(), copy.eval(), ref.eval())  # 2.0 2.0 2.0

print(tf.get_default_graph().get_operations())

You may wonder, why would anyone want to introduce a new op when she can simply make an assignment. There cases when assignment doesn't work, but tf.identity does, exactly because it creates a new op, e.g. in control flow. See this question: How to add control dependency to Tensorflow op.

回答2:

The accepted answer doesn't seem to be correct anymore, at least with eager execution. tf.identity will return a different tensor with the same values, so it is equivalent to <variable>.read_value(). Here is an example from the documentation, showing the current behavior:

a = tf.Variable(5)
a_identity = tf.identity(a)
a.assign_add(1)

a.numpy() # 6 

a_identity.numpy() # 5

Thus, we see that ops performed on a tensor returned by tf.identity do not affect the tensor on which it was called.

来源：https://stackoverflow.com/questions/49173854/tensorflow-what-is-the-difference-between-tf-identity-and-operator