TensorFlow v1.10+ load SavedModel with different device placement or manually set dynamic device placement?

Question

So in TensorFlow\'s guide for using GPUs there is a part about using multiple GPUs in a \"multi-tower fashion\":

...
for d in [\'/device:GPU:2\', \'/device:G         


        

Answer 1

I am doing some research on this topic recently and to my knowledge, your question 1 can work only if you clear all devices when you export the model in the original tensorflow code, with flag clear_devices=True.

In my own code, it looks like

builder = tf.saved_model.builder.SavedModelBuilder('osvos_saved')
builder.add_meta_graph_and_variables(sess, ['serve'], clear_devices=True)
builder.save()

If you only have a exported model, seems not possible. You can refer to this issue.

I'm currently trying to find a way to fix this, as stated in my stackoverflow question. Hope the workaround can help you.