问题
This question is about TensorFlow (and TensorBoard) version 2.2rc3, but I have experienced the same issue with 2.1. It is a continuation of the question 'Messed up TensorBoard graphs due to Python operations'.
Consider the following code:
from datetime import datetime
import tensorflow as tf
from tensorflow import keras
inputs = keras.layers.Input(shape=(784, ))
outputs = tf.zeros([32, 10], tf.float32)
for i in range(0, 3):
x = keras.layers.Dense(32, activation='relu', name='Model/Block' + str(i) + '/relu')(inputs)
x = keras.layers.Dropout(0.2, name='Model/Block' + str(i) + '/dropout')(x)
x = keras.layers.Dense(10, activation='softmax', name='Model/Block' + str(i) + '/softmax')(x)
outputs = keras.layers.Lambda(lambda x: x[0] + x[1], name='Model/add/add' + str(i))([outputs, x])
model = tf.keras.Model(inputs=inputs, outputs=outputs)
model.summary(line_length=100, positions=[.45, .58, .67, 1.])
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
x_train = x_train.reshape(60000, 784).astype('float32') / 255
x_test = x_test.reshape(10000, 784).astype('float32') / 255
model.compile(loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=keras.optimizers.RMSprop(),
metrics=['accuracy'])
logdir = "logs/" + datetime.now().strftime("%Y%m%d-%H%M%S")
tensorboard_callback = keras.callbacks.TensorBoard(log_dir=logdir)
model.fit(x_train, y_train,
batch_size=32,
epochs=5,
validation_split=0.2,
callbacks=[tensorboard_callback])
When run, it prints the following summary:
____________________________________________________________________________________
Layer (type) Output Shap Param Connected to
====================================================================================
input_1 (InputLayer) [(None, 784 0
____________________________________________________________________________________
Model/Block0/relu (Dense) (None, 32) 25120 input_1[0][0]
____________________________________________________________________________________
Model/Block0/dropout (Dropout) (None, 32) 0 Model/Block0/relu[0][0]
____________________________________________________________________________________
Model/Block1/relu (Dense) (None, 32) 25120 input_1[0][0]
____________________________________________________________________________________
Model/Block0/softmax (Dense) (None, 10) 330 Model/Block0/dropout[0][0]
____________________________________________________________________________________
Model/Block1/dropout (Dropout) (None, 32) 0 Model/Block1/relu[0][0]
____________________________________________________________________________________
Model/Block2/relu (Dense) (None, 32) 25120 input_1[0][0]
____________________________________________________________________________________
tf_op_layer_AddV2 (TensorFlowOpLayer) [(32, 10)] 0 Model/Block0/softmax[0][0]
____________________________________________________________________________________
Model/Block1/softmax (Dense) (None, 10) 330 Model/Block1/dropout[0][0]
____________________________________________________________________________________
Model/Block2/dropout (Dropout) (None, 32) 0 Model/Block2/relu[0][0]
____________________________________________________________________________________
Model/add/add1 (Lambda) (32, 10) 0 tf_op_layer_AddV2[0][0]
Model/Block1/softmax[0][0]
____________________________________________________________________________________
Model/Block2/softmax (Dense) (None, 10) 330 Model/Block2/dropout[0][0]
____________________________________________________________________________________
Model/add/add2 (Lambda) (32, 10) 0 Model/add/add1[0][0]
Model/Block2/softmax[0][0]
====================================================================================
Note the strange entry tf_op_layer_AddV2 (TensorFlowOpLayer). This kind of entry makes the TensorBoard graphs very messy. It turns out that when avoiding the use of the tf.zeros(), this strange tf_op_layer_AddV2 element is not added. So the following code will not generate any tf_op_layer element:
from datetime import datetime
import tensorflow as tf
from tensorflow import keras
inputs = keras.layers.Input(shape=(784, ))
x = keras.layers.Dense(32, activation='relu', name='Model/Block0/relu')(inputs)
x = keras.layers.Dropout(0.2, name='Model/Block0/dropout')(x)
outputs = keras.layers.Dense(10, activation='softmax', name='Model/Block0/softmax')(x)
for i in range(1, 3):
x = keras.layers.Dense(32, activation='relu', name='Model/Block' + str(i) + '/relu')(inputs)
x = keras.layers.Dropout(0.2, name='Model/Block' + str(i) + '/dropout')(x)
x = keras.layers.Dense(10, activation='softmax', name='Model/Block' + str(i) + '/softmax')(x)
outputs = keras.layers.Lambda(lambda x: x[0] + x[1], name='Model/add/add' + str(i))([outputs, x])
model = tf.keras.Model(inputs=inputs, outputs=outputs)
model.summary(line_length=84, positions=[.46, .60, .69, 1.])
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
x_train = x_train.reshape(60000, 784).astype('float32') / 255
x_test = x_test.reshape(10000, 784).astype('float32') / 255
model.compile(loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=keras.optimizers.RMSprop(),
metrics=['accuracy'])
logdir = "logs/" + datetime.now().strftime("%Y%m%d-%H%M%S")
tensorboard_callback = keras.callbacks.TensorBoard(log_dir=logdir)
model.fit(x_train, y_train,
batch_size=32,
epochs=5,
validation_split=0.2,
callbacks=[tensorboard_callback])
There are many more complex examples where the tf_op_layer_ elements are created. It will be appreciated if it is exaplained why they are created and how to avoid them.
来源：https://stackoverflow.com/questions/61594352/tensorflowoplayer-messes-up-the-tensorboard-graphs