tf.keras.predict() is much slower than Keras predict()

问题

When using the Keras that comes embedded with Tensorflow (Tensorflow 2), I noticed a severe increase in computational time when using the predict() function from the Keras embedded inside Tensorflow and the predict() from standalone Keras. See the toy code below:

import tensorflow
import keras
import numpy as np
import time

test = np.array([[0.1, 0.1, 0.1, 0.1, 0.1, 0.5, 0.1, 0., 0.1, 0.2]])

# Keras from inside Tensorflow
model_1 = tensorflow.keras.Sequential([
  tensorflow.keras.layers.Dense(1, activation='relu', input_shape=(10,)),
])

start_1 = time.time()
for i in range(1000):
    result = model_1.predict(test)
elapsed_time_1 = time.time() - start_1

# Standalone Keras
model_2 = keras.models.Sequential([
  keras.layers.Dense(1, activation='relu', input_shape=(10,)),
]) 

start_2 = time.time()
for i in range(1000):
    result = model_2.predict(test)
elapsed_time_2 = time.time() - start_2

print(elapsed_time_1)
print(elapsed_time_2)

The output from the code below in my machine is

17.82757878303528
0.31248927116394043

The expected output is that the predict() from tensorflow.keras should take the same amount of time for the same task, when compared to the predict() from standalone Keras.

My questions are:

1. Why is this happening?
2. How can I fix this?

My specs:

Python version: Python 3.6.8

Keras version: 2.3.1

Tensorflow version: 2.1.0

Running on Windows 10

回答1:

It's mostly due to eager execution. You can turn off eager execution with

tensorflow.compat.v1.disable_eager_execution()

Doing that, the tf.keras is still ~2x slower, which I'm not sure why, but not by orders of magnitude. Both of them go even faster if you convert to tensors beforehand.

来源：https://stackoverflow.com/questions/61698995/tf-keras-predict-is-much-slower-than-keras-predict