问题
Suppose I have an existing Windows application (hello1.exe). Suppose I also have a tf.Keras neural net model trained and classifying test data. How do I take the latter "out of the lab" and deploy it in the "real world" by combining the two components something like this:
> hello2.exe cat.jpg
Hello World. BTW: the file 'cat.jpg' contains a cat
That is, hello2.exe extends my existing hello1.exe by using the trained model to infer the contents of the input file. I don't mind if the input data is not an image - I just want to understand the principles of integrating any Tensorflow/Keras model into a Windows application and using it for inference. So the model can be as deep or shallow as you care to imagine.
Important: speed of invoking the model is critical - serving the model is not an option; it must be integrated INTO the executable.
When I tried this 2 years ago (albeit native TF, not Keras), it was very hard. It is not clear if it has become any easier. Some of the issues I think a working example would help provide guidance on are:
- Does Tensorflow require a 64-bit application, meaning that if my existing application is 32-bit, it will need upgrading to a 64-bit application so that it can link to TF libraries?
- Will I need to statically link to TF functionality, use DLLs, or is it my choice? Will I need to build Tensorflow from source or is that my choice too?
- Should I consider Tensorflow Lite? It looks much simpler, but it uses interpreted models - what are the performance implications?
- Do I have decent control over threading issues if my existing application is already multi-threaded and/or real-time?
- Once I have something vaguely working, what options do I have for optimising performance if I need it?
I will be trying to answer this question myself - please feel free to beat me to it!
来源：https://stackoverflow.com/questions/61812992/how-to-integrate-tensorflow-keras-model-into-windows-application-for-fast-infe