Is Keras thread safe?

Question

I\'m using Python and Keras (currently using Theano backend, but I have no qualms with switching). I have a neural network that I load and process multiple sources of inform

Answer 1

to quote the kind fcholet:

_make_predict_function is a private API. We should not recommend calling it.

Here, the user should simply call predict first.

Note that Keras models can't be guaranteed to be thread-safe. Consider having independent copies of the model in each thread for CPU inference.

Answer 2

Yes, Keras is thread safe, if you pay a little attention to it.

In fact, in reinforcement learning there is an algorithm called Asynchronous Advantage Actor Critics (A3C) where each agent relies on the same neural network to tell them what they should do in a given state. In other words, each thread calls model.predict concurrently as in your problem. An example implementation with Keras of it is here.

You should, however, pay extra attention to this line if you looked into the code: model._make_predict_function() # have to initialize before threading

This is never mentioned in the Keras docs, but its necessary to make it work concurrently. In short, _make_predict_function is a function that compiles the predict function. In multi thread setting, you have to manually call this function to compile predict in advance, otherwise the predict function will not be compiled until you run it the first time, which will be problematic when many threading calling it at once. You can see a detailed explanation here.

I have not met any other issues with multi threading in Keras till now.