Tensorflow: When use tf.expand_dims?

Question

Tensorflow tutorials include the use of tf.expand_dims to add a \"batch dimension\" to a tensor. I have read the docs for this function but it still is rather m

Answer 1

To add to Da Tong's answer, you may want to expand more than one dimension at the same time. For instance, if you are performing TensorFlow's conv1d operation on vectors of rank 1, you need to feed them with rank three.

Performing expand_dims several times is readable, but might introduce some overhead into the computational graph. You can get the same functionality in a one-liner with reshape:

import tensorflow as tf

# having some tensor of rank 1, it could be an audio signal, a word vector...
tensor = tf.ones(100)
print(tensor.get_shape()) # => (100,)

# expand its dimensionality to fit into conv2d
tensor_expand = tf.expand_dims(tensor, 0)
tensor_expand = tf.expand_dims(tensor_expand, 0)
tensor_expand = tf.expand_dims(tensor_expand, -1)
print(tensor_expand.get_shape()) # => (1, 1, 100, 1)

# do the same in one line with reshape
tensor_reshape = tf.reshape(tensor, [1, 1, tensor.get_shape().as_list()[0],1])
print(tensor_reshape.get_shape()) # => (1, 1, 100, 1)

NOTE: In case you get the error TypeError: Failed to convert object of type <type 'list'> to Tensor., try to pass tf.shape(x)[0] instead of x.get_shape()[0] as suggested here.

Hope it helps!
Cheers,
Andres

Answer 2

expand_dims will not add or reduce elements in a tensor, it just changes the shape by adding 1 to dimensions. For example, a vector with 10 elements could be treated as a 10x1 matrix.

The situation I have met to use expand_dims is when I tried to build a ConvNet to classify grayscale images. The grayscale images will be loaded as matrix of size [320, 320]. However, tf.nn.conv2d require input to be [batch, in_height, in_width, in_channels], where the in_channels dimension is missing in my data which in this case should be 1. So I used expand_dims to add one more dimension.

In your case, I do not think you need expand_dims.