Why does tf.executing_eagerly() return False in TensorFlow 2?

问题

Let me explain my set up. I am using TensorFlow 2.1, the Keras version shipped with TF, and TensorFlow Probability 0.9.

I have a function get_model that creates (with the functional API) and returns a model using Keras and custom layers. In the __init__ method of these custom layers A, I call a method A.m, which executes the statement print(tf.executing_eagerly()), but it returns False. Why?

To be more precise, this is roughly my setup

def get_model():
    inp = Input(...)
    x = A(...)(inp) 
    x = A(...)(x)
    ...
    model = Model(inp, out)
    model.compile(...)
    return model

class A(tfp.layers.DenseFlipout): # TensorFlow Probability
    def __init__(...):
        self.m()

    def m(self): 
        print(tf.executing_eagerly()) # Prints False

The documentation of tf.executing_eagerly says

Eager execution is enabled by default and this API returns True in most of cases. However, this API might return False in the following use cases.

• Executing inside tf.function, unless under tf.init_scope or tf.config.experimental_run_functions_eagerly(True) is previously called.
• Executing inside a transformation function for tf.dataset.
• tf.compat.v1.disable_eager_execution() is called.

But these cases are not my case, so tf.executing_eagerly() should return True in my case, but no. Why?

Here's a simple complete example (in TF 2.1) that illustrates the problem.

import tensorflow as tf


class MyLayer(tf.keras.layers.Layer):
    def call(self, inputs):
        tf.print("tf.executing_eagerly() =", tf.executing_eagerly())
        return inputs


def get_model():
    inp = tf.keras.layers.Input(shape=(1,))
    out = MyLayer(8)(inp)
    model = tf.keras.Model(inputs=inp, outputs=out)
    model.summary()
    return model


def train():
    model = get_model()
    model.compile(optimizer="adam", loss="mae")
    x_train = [2, 3, 4, 1, 2, 6]
    y_train = [1, 0, 1, 0, 1, 1]
    model.fit(x_train, y_train)


if __name__ == '__main__':
    train()

This example prints tf.executing_eagerly() = False.

See the related Github issue.

回答1:

As far as I know, when an input to a custom layer is symbolic input, then the layer is executed in graph (non-eager) mode. However, if your input to the custom layer is an eager tensor (as in the following example #1, then the custom layer is executed in the eager mode. So your model's output tf.executing_eagerly() = False is expected.

Example #1

from tensorflow.keras import layers


class Linear(layers.Layer):

  def __init__(self, units=32, input_dim=32):
    super(Linear, self).__init__()
    w_init = tf.random_normal_initializer()
    self.w = tf.Variable(initial_value=w_init(shape=(input_dim, units),
                                              dtype='float32'),
                         trainable=True)
    b_init = tf.zeros_initializer()
    self.b = tf.Variable(initial_value=b_init(shape=(units,),
                                              dtype='float32'),
                         trainable=True)

  def call(self, inputs):
    print("tf.executing_eagerly() =", tf.executing_eagerly())
    return tf.matmul(inputs, self.w) + self.b

x = tf.ones((1, 2)) # returns tf.executing_eagerly() = True
#x = tf.keras.layers.Input(shape=(2,)) #tf.executing_eagerly() = False
linear_layer = Linear(4, 2)
y = linear_layer(x)
print(y) 
#output in graph mode: Tensor("linear_9/Identity:0", shape=(None, 4), dtype=float32)
#output in Eager mode: tf.Tensor([[-0.03011466  0.02563028  0.01234017  0.02272708]], shape=(1, 4), dtype=float32)

Here is another example with Keras functional API where custom layer was used (similar to you). This model is executed in graph mode and prints tf.executing_eagerly() = False as in your case.

from tensorflow import keras
from tensorflow.keras import layers
class CustomDense(layers.Layer):
  def __init__(self, units=32):
    super(CustomDense, self).__init__()
    self.units = units

  def build(self, input_shape):
    self.w = self.add_weight(shape=(input_shape[-1], self.units),
                             initializer='random_normal',
                             trainable=True)
    self.b = self.add_weight(shape=(self.units,),
                             initializer='random_normal',
                             trainable=True)

  def call(self, inputs):
    print("tf.executing_eagerly() =", tf.executing_eagerly())
    return tf.matmul(inputs, self.w) + self.b


inputs = keras.Input((4,))
outputs = CustomDense(10)(inputs)

model = keras.Model(inputs, outputs) 

来源：https://stackoverflow.com/questions/61355474/why-does-tf-executing-eagerly-return-false-in-tensorflow-2