Post-training full integer quantization of Keras model

问题

I'm trying to do post-training full 8-bit quantization of a Keras model to compile and deploy to EdgeTPU. I have a trained Keras model saved as .h5 file, and am trying to go through the steps as specified here: https://coral.withgoogle.com/docs/edgetpu/models-intro/, for deployment to the Coral Dev Board.

I'm following these instructions for quantization: https://www.tensorflow.org/lite/performance/post_training_quantization#full_integer_quantization_of_weights_and_activations)

I’m trying to use the following code:

import tensorflow as tf

num_calibration_steps = 100 
def representative_dataset_gen():
  for _ in range(num_calibration_steps):
    # Get sample input data as a numpy array in a method of your choosing.
    yield [X_train_quant_conv]

converter = tf.compat.v1.lite.TFLiteConverter.from_keras_model_file('/tmp/classNN_simple.h5')
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8
converter.representative_dataset = representative_dataset_gen
tflite_full_integer_quant_model = converter.convert()

where X_train_quant_conv is a subset of my training data converted to np.array and of type np.float32

When running this piece of code, I get the following error: ValueError: Cannot set tensor: Dimension mismatch

I’ve tried changing the function representative_dataset_gen() in different ways, but every time I get a new error. I’m not sure how this function should be. I’m also in doubt of what value num_calibration_steps should have.

Any suggestions or working examples are very appreciated.

This question is very similar to this answered question: Convert Keras model to quantized Tensorflow Lite model that can be used on Edge TPU

回答1:

You might want to look at my demo script for quantization, on github.

It's just a guess since I can't see what X_train_quant_conv really is, but in my working demo, I yield one image at a time (random data created on the fly, in my case) in representative_dataset_gen(). The image is stored as batch of size 1 (e.g., tensor shape is (1, 56, 56, 32) for my 52x52x32 image). There are 32 channels, though there would typically just be 3, for a color image. I think representative_dataset_gen() has to yield a list containing a tensor (or more than one?) for which the first dimension is of length 1.

image_shape = (56, 56, 32)

def representative_dataset_gen():
    num_calibration_images = 10
    for i in range(num_calibration_images):
        image = tf.random.normal([1] + list(image_shape))
        yield [image]

来源：https://stackoverflow.com/questions/58623192/post-training-full-integer-quantization-of-keras-model