问题
Okay so in my app i am trying to implement face recognition using face net model which is converted to tflite averaging at about 93 MB approximately, however this model eventually increases size of my apk. so i am trying to find alternate ways to deal with this
Firstly i can think of is to compress it in some way and then uncompress when app is installed
Another way is that i should upload that model to server and after being downloaded get it loaded in my application. However i do not seem know how to implement this:
By default face net allows implementation from assets folder
var facenet = FaceNet(getAssets());
But in case i'm downloading that model how can i get it loaded in my application?
Here is my face net intilization code:
public FaceNet(AssetManager assetManager) throws IOException {
tfliteModel = loadModelFile(assetManager);
tflite = new Interpreter(tfliteModel, tfliteOptions);
imgData = ByteBuffer.allocateDirect(
BATCH_SIZE
* IMAGE_HEIGHT
* IMAGE_WIDTH
* NUM_CHANNELS
* NUM_BYTES_PER_CHANNEL);
imgData.order(ByteOrder.nativeOrder());
}
private MappedByteBuffer loadModelFile(AssetManager assetManager) throws IOException {
AssetFileDescriptor fileDescriptor = assetManager.openFd(MODEL_PATH);
FileInputStream inputStream = new FileInputStream(fileDescriptor.getFileDescriptor());
FileChannel fileChannel = inputStream.getChannel();
long startOffset = fileDescriptor.getStartOffset();
long declaredLength = fileDescriptor.getDeclaredLength();
return fileChannel.map(FileChannel.MapMode.READ_ONLY, startOffset, declaredLength);
}
My FaceNet Class:
public class FaceNet {
private static final String MODEL_PATH = "facenet.tflite";
private static final float IMAGE_MEAN = 127.5f;
private static final float IMAGE_STD = 127.5f;
private static final int BATCH_SIZE = 1;
private static final int IMAGE_HEIGHT = 160;
private static final int IMAGE_WIDTH = 160;
private static final int NUM_CHANNELS = 3;
private static final int NUM_BYTES_PER_CHANNEL = 4;
private static final int EMBEDDING_SIZE = 512;
private final int[] intValues = new int[IMAGE_HEIGHT * IMAGE_WIDTH];
private ByteBuffer imgData;
private MappedByteBuffer tfliteModel;
private Interpreter tflite;
private final Interpreter.Options tfliteOptions = new Interpreter.Options();
public FaceNet(AssetManager assetManager) throws IOException {
tfliteModel = loadModelFile(assetManager);
tflite = new Interpreter(tfliteModel, tfliteOptions);
imgData = ByteBuffer.allocateDirect(
BATCH_SIZE
* IMAGE_HEIGHT
* IMAGE_WIDTH
* NUM_CHANNELS
* NUM_BYTES_PER_CHANNEL);
imgData.order(ByteOrder.nativeOrder());
}
private MappedByteBuffer loadModelFile(AssetManager assetManager) throws IOException {
AssetFileDescriptor fileDescriptor = assetManager.openFd(MODEL_PATH);
FileInputStream inputStream = new FileInputStream(fileDescriptor.getFileDescriptor());
FileChannel fileChannel = inputStream.getChannel();
long startOffset = fileDescriptor.getStartOffset();
long declaredLength = fileDescriptor.getDeclaredLength();
return fileChannel.map(FileChannel.MapMode.READ_ONLY, startOffset, declaredLength);
}
private void convertBitmapToByteBuffer(Bitmap bitmap) {
if (imgData == null) {
return;
}
imgData.rewind();
bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0, bitmap.getWidth(), bitmap.getHeight());
// Convert the image to floating point.
int pixel = 0;
for (int i = 0; i < IMAGE_HEIGHT; ++i) {
for (int j = 0; j < IMAGE_WIDTH; ++j) {
final int val = intValues[pixel++];
addPixelValue(val);
}
}
}
private void addPixelValue(int pixelValue){
//imgData.putFloat((((pixelValue >> 16) & 0xFF) - IMAGE_MEAN) / IMAGE_STD);
//imgData.putFloat((((pixelValue >> 8) & 0xFF) - IMAGE_MEAN) / IMAGE_STD);
//imgData.putFloat(((pixelValue & 0xFF) - IMAGE_MEAN) / IMAGE_STD);
imgData.putFloat(((pixelValue >> 16) & 0xFF) / 255.0f);
imgData.putFloat(((pixelValue >> 8) & 0xFF) / 255.0f);
imgData.putFloat((pixelValue & 0xFF) / 255.0f);
}
public void inspectModel(){
String tag = "Model Inspection";
Log.i(tag, "Number of input tensors: " + String.valueOf(tflite.getInputTensorCount()));
Log.i(tag, "Number of output tensors: " + String.valueOf(tflite.getOutputTensorCount()));
Log.i(tag, tflite.getInputTensor(0).toString());
Log.i(tag, "Input tensor data type: " + tflite.getInputTensor(0).dataType());
Log.i(tag, "Input tensor shape: " + Arrays.toString(tflite.getInputTensor(0).shape()));
Log.i(tag, "Output tensor 0 shape: " + Arrays.toString(tflite.getOutputTensor(0).shape()));
}
private Bitmap resizedBitmap(Bitmap bitmap, int height, int width){
return Bitmap.createScaledBitmap(bitmap, width, height, true);
}
private Bitmap croppedBitmap(Bitmap bitmap, int upperCornerX, int upperCornerY, int height, int width){
return Bitmap.createBitmap(bitmap, upperCornerX, upperCornerY, width, height);
}
private float[][] run(Bitmap bitmap){
bitmap = resizedBitmap(bitmap, IMAGE_HEIGHT, IMAGE_WIDTH);
convertBitmapToByteBuffer(bitmap);
float[][] embeddings = new float[1][512];
tflite.run(imgData, embeddings);
return embeddings;
}
public double getSimilarityScore(Bitmap face1, Bitmap face2){
float[][] face1_embedding = run(face1);
float[][] face2_embedding = run(face2);
double distance = 0.0;
for (int i = 0; i < EMBEDDING_SIZE; i++){
distance += (face1_embedding[0][i] - face2_embedding[0][i]) * (face1_embedding[0][i] - face2_embedding[0][i]);
}
distance = Math.sqrt(distance);
return distance;
}
public void close(){
if (tflite != null) {
tflite.close();
tflite = null;
}
tfliteModel = null;
}
}
回答1:
Well i can't think of any solution for reducing the size of your model file but by observing your class i can say that after all it's returning a mapped byte buffer from your file input stream so to get file from storage simply put your file in facenet folder in external storage and then get mapped bytebuffer on your file input stream here is a solution in kotlin.
class FaceNetStorage @Throws(IOException::class)
constructor() {
private val intValues = IntArray(IMAGE_HEIGHT * IMAGE_WIDTH)
private var imgData: ByteBuffer? = null
private var tfliteModel: MappedByteBuffer? = null
private var tflite: Interpreter? = null
private val tfliteOptions = Interpreter.Options()
init {
val str = Environment.getExternalStorageDirectory().toString()+"/Facenet"
val sd_main = File(str)
var success = true
if (!sd_main.exists()) {
success = sd_main.mkdir()
}
if (success) {
val sd = File(str+"/"+MODEL_PATH)
tfliteModel = loadModelFile(sd)
tflite = Interpreter(tfliteModel!!, tfliteOptions)
imgData = ByteBuffer.allocateDirect(
BATCH_SIZE
* IMAGE_HEIGHT
* IMAGE_WIDTH
* NUM_CHANNELS
* NUM_BYTES_PER_CHANNEL)
imgData!!.order(ByteOrder.nativeOrder())
}
}
@Throws(IOException::class)
private fun loadModelFile(file: File): MappedByteBuffer {
val inputStream = FileInputStream(file)
val fileChannel = inputStream.channel
return fileChannel.map(FileChannel.MapMode.READ_ONLY, 0, fileChannel.size())
}
private fun convertBitmapToByteBuffer(bitmap: Bitmap) {
if (imgData == null) {
return
}
imgData!!.rewind()
bitmap.getPixels(intValues, 0, bitmap.width, 0, 0, bitmap.width, bitmap.height)
// Convert the image to floating point.
var pixel = 0
for (i in 0 until IMAGE_HEIGHT) {
for (j in 0 until IMAGE_WIDTH) {
val `val` = intValues[pixel++]
addPixelValue(`val`)
}
}
}
private fun addPixelValue(pixelValue: Int) {
imgData!!.putFloat((pixelValue shr 16 and 0xFF) / 255.0f)
imgData!!.putFloat((pixelValue shr 8 and 0xFF) / 255.0f)
imgData!!.putFloat((pixelValue and 0xFF) / 255.0f)
}
fun inspectModel() {
val tag = "Model Inspection"
Log.i(tag, "Number of input tensors: " + tflite!!.inputTensorCount.toString())
Log.i(tag, "Number of output tensors: " + tflite!!.outputTensorCount.toString())
Log.i(tag, tflite!!.getInputTensor(0).toString())
Log.i(tag, "Input tensor data type: " + tflite!!.getInputTensor(0).dataType())
Log.i(tag, "Input tensor shape: " + Arrays.toString(tflite!!.getInputTensor(0).shape()))
Log.i(tag, "Output tensor 0 shape: " + Arrays.toString(tflite!!.getOutputTensor(0).shape()))
}
private fun resizedBitmap(bitmap: Bitmap, height: Int, width: Int): Bitmap {
return Bitmap.createScaledBitmap(bitmap, width, height, true)
}
private fun croppedBitmap(bitmap: Bitmap, upperCornerX: Int, upperCornerY: Int, height: Int, width: Int): Bitmap {
return Bitmap.createBitmap(bitmap, upperCornerX, upperCornerY, width, height)
}
private fun run(bitmap: Bitmap): Array<FloatArray> {
var bitmap = bitmap
bitmap = resizedBitmap(bitmap, IMAGE_HEIGHT, IMAGE_WIDTH)
convertBitmapToByteBuffer(bitmap)
val embeddings = Array(1) { FloatArray(512) }
tflite!!.run(imgData, embeddings)
return embeddings
}
fun getSimilarityScore(face1: Bitmap, face2: Bitmap): Double {
val face1_embedding = run(face1)
val face2_embedding = run(face2)
var distance = 0.0
for (i in 0 until EMBEDDING_SIZE) {
distance += ((face1_embedding[0][i] - face2_embedding[0][i]) * (face1_embedding[0][i] - face2_embedding[0][i])).toDouble()
}
distance = Math.sqrt(distance)
return distance
}
fun close() {
if (tflite != null) {
tflite!!.close()
tflite = null
}
tfliteModel = null
}
companion object {
private val MODEL_PATH = "facenet.tflite"
private val IMAGE_MEAN = 127.5f
private val IMAGE_STD = 127.5f
private val BATCH_SIZE = 1
private val IMAGE_HEIGHT = 160
private val IMAGE_WIDTH = 160
private val NUM_CHANNELS = 3
private val NUM_BYTES_PER_CHANNEL = 4
private val EMBEDDING_SIZE = 512
}
}
回答2:
I'd recommend quantizing your model. This would reduce the file size by about 1/4. You can try just weight quantization, or full quantization.
Using the Python API, for only weight quantization:
import tensorflow as tf
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
converter.optimizations = [tf.lite.Optimize.OPTIMIZE_FOR_SIZE]
tflite_quant_model = converter.convert()
For full quantization, I'd recommend using a representative dataset to reduce the accuracy loss associated with quantization.
import tensorflow as tf
def representative_dataset_gen():
for _ in range(num_calibration_steps):
# Get sample input data as a numpy array in a method of your choosing.
yield [input]
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset_gen
tflite_quant_model = converter.convert()
You can also try mobilenet architectures. Quantized versions of these can be anywhere from <1MB to ~5MB. You can easily find some Tensorflow implentations of mobilefacenets with a quick google search, but here's a link to the paper that started it: https://arxiv.org/abs/1804.07573
来源：https://stackoverflow.com/questions/59048118/how-to-reduce-size-of-tflite-model-or-download-and-set-it-programmatically