What's the difference between tf.placeholder and tf.Variable?

Question

I\'m a newbie to TensorFlow. I\'m confused about the difference between tf.placeholder and tf.Variable. In my view, tf.placeholder is

Answer 1

Adding to other's answers, they also explain it very well in this MNIST tutorial on Tensoflow website:

We describe these interacting operations by manipulating symbolic variables. Let's create one:

x = tf.placeholder(tf.float32, [None, 784]),

x isn't a specific value. It's a placeholder, a value that we'll input when we ask TensorFlow to run a computation. We want to be able to input any number of MNIST images, each flattened into a 784-dimensional vector. We represent this as a 2-D tensor of floating-point numbers, with a shape [None, 784]. (Here None means that a dimension can be of any length.)

We also need the weights and biases for our model. We could imagine treating these like additional inputs, but TensorFlow has an even better way to handle it: Variable. A Variable is a modifiable tensor that lives in TensorFlow's graph of interacting operations. It can be used and even modified by the computation. For machine learning applications, one generally has the model parameters be Variables.

W = tf.Variable(tf.zeros([784, 10]))

b = tf.Variable(tf.zeros([10]))

We create these Variables by giving tf.Variable the initial value of the Variable: in this case, we initialize both W and b as tensors full of zeros. Since we are going to learn W and b, it doesn't matter very much what they initially are.

Answer 2

Tensorflow 2.0 Compatible Answer: The concept of Placeholders, tf.placeholder will not be available in Tensorflow 2.x (>= 2.0) by default, as the Default Execution Mode is Eager Execution.

However, we can use them if used in Graph Mode (Disable Eager Execution).

Equivalent command for TF Placeholder in version 2.x is tf.compat.v1.placeholder.

Equivalent Command for TF Variable in version 2.x is tf.Variable and if you want to migrate the code from 1.x to 2.x, the equivalent command is

tf.compat.v2.Variable.

Please refer this Tensorflow Page for more information about Tensorflow Version 2.0.

Please refer the Migration Guide for more information about migration from versions 1.x to 2.x.

Answer 3

In short, you use tf.Variable for trainable variables such as weights (W) and biases (B) for your model.

weights = tf.Variable(
    tf.truncated_normal([IMAGE_PIXELS, hidden1_units],
                    stddev=1.0 / math.sqrt(float(IMAGE_PIXELS))), name='weights')

biases = tf.Variable(tf.zeros([hidden1_units]), name='biases')

tf.placeholder is used to feed actual training examples.

images_placeholder = tf.placeholder(tf.float32, shape=(batch_size, IMAGE_PIXELS))
labels_placeholder = tf.placeholder(tf.int32, shape=(batch_size))

This is how you feed the training examples during the training:

for step in xrange(FLAGS.max_steps):
    feed_dict = {
       images_placeholder: images_feed,
       labels_placeholder: labels_feed,
     }
    _, loss_value = sess.run([train_op, loss], feed_dict=feed_dict)

Your tf.variables will be trained (modified) as the result of this training.

See more at https://www.tensorflow.org/versions/r0.7/tutorials/mnist/tf/index.html. (Examples are taken from the web page.)

Answer 4

Think of a computation graph. In such graph, we need an input node to pass our data to the graph, those nodes should be defined as Placeholder in tensorflow.

Do not think as a general program in Python. You can write a Python program and do all those stuff that guys explained in other answers just by Variables, but for computation graphs in tensorflow, to feed your data to the graph, you need to define those nods as Placeholders.

Answer 5

Tensorflow uses three types of containers to store/execute the process

1. Constants :Constants holds the typical data.

2. variables: Data values will be changed, with respective the functions such as cost_function..

3. placeholders: Training/Testing data will be passed in to the graph.

Answer 6

Think of Variable in tensorflow as a normal variables which we use in programming languages. We initialize variables, we can modify it later as well. Whereas placeholder doesn’t require initial value. Placeholder simply allocates block of memory for future use. Later, we can use feed_dict to feed the data into placeholder. By default, placeholder has an unconstrained shape, which allows you to feed tensors of different shapes in a session. You can make constrained shape by passing optional argument -shape, as I have done below.

x = tf.placeholder(tf.float32,(3,4))
y =  x + 2

sess = tf.Session()
print(sess.run(y)) # will cause an error

s = np.random.rand(3,4)
print(sess.run(y, feed_dict={x:s}))

While doing Machine Learning task, most of the time we are unaware of number of rows but (let’s assume) we do know the number of features or columns. In that case, we can use None.

x = tf.placeholder(tf.float32, shape=(None,4))

Now, at run time we can feed any matrix with 4 columns and any number of rows.

Also, Placeholders are used for input data ( they are kind of variables which we use to feed our model), where as Variables are parameters such as weights that we train over time.