What is the difference between tf.initialize_all_variables() and tf.initialize_local_variables()?

Question

I am reviewing the code in this example: fully_connected_reader.py

I am confused with Line 147 and 148:

init_op = tf.group(tf.initialize_all_variable         


        

Answer 1

A local variable in TF is any variable which was created with collections=[tf.GraphKeys.LOCAL_VARIABLES]. For example:

e = tf.Variable(6, name='var_e', collections=[tf.GraphKeys.LOCAL_VARIABLES])

LOCAL_VARIABLES: the subset of Variable objects that are local to each machine. Usually used for temporarily variables, like counters. Note: use tf.contrib.framework.local_variable to add to this collection.

They are usually not saved/restored to checkpoint and used for temporary or intermediate values. For a more detailed answer, take a look here.

A global variable is mostly every other variable initialized by you.

---

In a new version of TF you should use tf.global_variables_initializer(), tf.local_variables_initializer(), because the previous functions were deprecated.

Answer 2

tf.initialize_all_variables() is a shortcut to tf.initialize_variables(tf.all_variables()), tf.initialize_local_variables() is a shortcut to tf.initialize_variables(tf.local_variables()), which initializes variables in GraphKeys.VARIABLES and GraphKeys.LOCAL_VARIABLE collections, respectively.

Variables in GraphKeys.LOCAL_VARIABLES collection are variables that are added to the graph, but not saved or restored (source).

tf.Variable() by default adds a new variable to GraphKeys.VARIABLE collection, which can be controlled by collections= argument.

Answer 3

• GLOBAL_VARIABLES

Key to collect Variable objects that are global (shared across machines). Default collection for all variables, except local ones.

• LOCAL_VARIABLES

Key to collect local variables that are local to the machine and are not saved/restored.