Install Tensorflow with Quantization Support

Question

This is a follow-up of another question by me : Error with 8-bit Quantization in Tensorflow

Basically, I would like to install the Tensorflow with 8-bit quantization support. Currently, I installed Tensorflow 0.9 with pip installation method on CentOS 7 machine (without GPU support).

I could compile and run the codes as given in Pete Warden's blog post. But, I can't import the functions given in Pete Warden's reply. I would like to add the quantization support. I couldn't find any details about the quantization part in the Tensorflow documentation also.

Can anybody share the details on how to do it?

Answer 1

For time-being, I could figure out a method to do this. But still waiting for official method from any TensorFlow developers.

1. First install the tensorflow ( I tried both source installation as well as PIP installation, both are fine)
2. Get the tensorflow source from the Github repo and go to the tensorflow root directory (I would call it tensorflow_root.
3. Now compile the quantization script as given in Pete Warden's blog

bazel build tensorflow/contrib/quantization/tools:quantize_graph

This wil create ops libraries for quantized versions. Go to tensorflow_root/bazel-bin/tensorflow/contrib/quantization and you should see two library files : _quantized_ops.so and kernels/_quantized_kernels.so

4. Now in your script, along with tensorflow, you should import these two library files also, using a dedicated tensorflow function

You can do it using tf.load_op_library() function

import tensorflow as tf
qops = tf.load_op_library('[tensorflow_root]/bazel-bin/tensorflow/contrib/quantization/_quantized_ops.so')
qkernelops = tf.load_op_library('[tensorflow_root]/bazel-bin/tensorflow/contrib/quantization/kernels/_quantized_kernels.so')

Answer 2

Are you using GPU or CPU version of tensorflow.

for example it does not work on CPU, though as Abid has mentioned loading the ops libraries from local working directory.

But what is the point of using bazel build when latest version has these ops integrated and being imported without error.

$ khemant@saturn:~/DeepLearning/TF$ python -c "import tensorflow as tf; print(tf.__version__)"

0.12.0-rc0

$ khemant@saturn:~/DeepLearning/TF$ python

Python 2.7.6 (default, Oct 26 2016, 20:30:19) [GCC 4.8.4] on linux2 Type "help", "copyright", "credits" or "license" for more information.

>>> from tensorflow.contrib.quantization import load_quantized_ops_so

Traceback (most recent call last): File "", line 1, in ImportError: cannot import name load_quantized_ops_so

>>> import tensorflow as tf

>>> from tensorflow.contrib.quantization import load_quantized_ops_so

Traceback (most recent call last): File "", line 1, in ImportError: cannot import name load_quantized_ops_so `