How to get intermediate outputs in TF 2.3 Eager with learning_phase?

Question

Example below works in 2.2; K.function is changed significantly in 2.3, now building a Model in Eager execution, so we\'re passing Model(inputs=[

Answer 1

For fetching output of an intermediate layer in eager mode it's not necessary to build a K.function and use learning phase. Instead, we can build a model to achieve that:

partial_model = Model(model.inputs, model.layers[1].output)

x = np.random.rand(...)
output_train = partial_model([x], training=True)   # runs the model in training mode
output_test = partial_model([x], training=False)   # runs the model in test mode

Alternatively, if you insist on using K.function and want to toggle learning phase in eager mode, you can use eager_learning_phase_scope from tensorflow.python.keras.backend (note that this module is a superset of tensorflow.keras.backend and contains internal functions, such as the mentioned one, which may change in future versions):

from tensorflow.python.keras.backend import eager_learning_phase_scope

fn = K.function([model.input], [model.layers[1].output])

# run in test mode, i.e. 0 means test
with eager_learning_phase_scope(value=0):
    output_test = fn([x])

# run in training mode, i.e. 1 means training
with eager_learning_phase_scope(value=1):
    output_train = fn([x])