问题
Similar Posts: Firstly, these 2 posts are similar if not the same. I tried to implement these in vain. So I'm missing something probably because of my inexperience in Keras. similar 1 , similar 2
The problem: I have a data generator to feed data into various models to evaluate model performance and to learn Keras.
model.fit_generator(generator=img_gen.next_train(), ....
One of the inputs produced by this generator is a tensor "labels" of shape=[batch_size, num_letters]. This tensor is the first input to
K.ctc_batch_cost(labels, ....
similar to image_ocr.py line 369.
The above "keras example" creates an RNN/GRU where each output step of the RNN is one input to the ctc, where there are num_letters steps in the RNN and the labels is of shape (?, num_letters). This worked fine for the first 6 models I have tested so far.
I am testing a new model where each step of the RNN/GRU output is "1 to n" inputs to the ctc and leaving it to training to optimize the output of each step. So my ctc needs as it's first input a tensor of shape=(?,num_letters*n) but the data generator produces shape=(?, num_letters).
Side note: The RNN in my model actually produce as a whole shape=(?, n, num_letters). I know how to convert this to (?, n*num_letters).
Solution 1 is a hack, change the generator so that it's unique for this model being tested. This generator will produce tensors of shape=(?,num_letters*n). I don't like this because the generator is also being evaluated and would like it to be constant for each model being evaluated.
Solution 2 is a hack, create a generator that encapsulates the original generator and augments the produced output.
Solution 3: have the model take an input of shape=(?, num_letters) and concatenate the necessary padding so the shape is (?,num_letters*n) that the ctc wants.
Here is what I tried:
num_letters = 3
n = 5
keras_tensor_input = keras.layers.Input( shape=[num_letters], dtype='float32' )
print("keras_tensor_input = ", keras_tensor_input)
# keras_tensor_input = Tensor("input_1:0", shape=(?, 3), dtype=float32)
# note ? is batch size
keras_tensor_neg_1 = keras.backend.constant( -1, dtype='float32', shape=[num_letters] )
print("keras_tensor_neg_1 = ", keras_tensor_neg_1)
# keras_tensor_neg_1 = Tensor("Const_1:0", shape=(3,), dtype=float32)
# note no batch size
keras_tensor_neg_1_tiled = keras.backend.tile(keras_tensor_neg_1, (n-1))
print("keras_tensor_neg_1_tiled = ", keras_tensor_neg_1_tiled)
# keras_tensor_neg_1_tiled = Tensor("Tile_2:0", shape=(12,), dtype=float32)
# note no batch size, but now the correct fill length
# FAILED attempt to put in a batch size
layer_const_neg_1 = keras.layers.Lambda(lambda x: keras_tensor_neg_1_tiled ,output_shape=[(n-1)*num_letters] )
keras_tensor_neg_1_prime = layer_const_neg_1(keras_tensor_neg_1)
print("keras_tensor_neg_1_prime = ", keras_tensor_neg_1_prime)
# keras_tensor_neg_1_prime = Tensor("Tile_2:0", shape=(12,), dtype=float32)
# CRASH AT NEXT STEP BECAUSE NO ? in keras_tensor_neg_1_prime
# concatenate the input from the generator and the padding
keras_tensor_concat = keras.layers.Concatenate()( inputs = [keras_tensor_input, keras_tensor_neg_1_prime] )
print("keras_tensor_concat = ", keras_tensor_concat)
my_model = keras.models.Model( inputs=[keras_tensor_input], output=keras_tensor_concat)
# dummy optimizer, loss, and metric just to allow compile to pass
my_model.compile(optimizer='rmsprop',loss='categorical_crossentropy', metrics=['accuracy'])
# a batch of size 1, for a tensor of shape =[3]
d1 = numpy.array([[1, 2, 3]])
out = my_model.predict( [ d1 ] )
print(out)
Notes:
- could have made constant shape=[num_letters*(n-1)] and drop the tile, but the same problem of missing batch size remains.
- if I place batch_size as first dimensions, then it still fails complaining of (?, 3) can not be concatenated with (1, 12)
Thank you in advance.
回答1:
I found the solution for a model that takes in
# a batch of size 1, for a tensor of shape =[3]
d1 = numpy.array([[1, 2, 3]])
and produces as output
out = my_model.predict( [ d1 ] )
print(out)
# [[ 1. 2. 3. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1.]]
The big lesson is that keras.layers want as input, the output of other keras.layers. So functionality from keras.backend like tile must be wrapped in keras.layers.Lambda before feeding them as input to keras.layer.
Thank you to
- Stackoverflow: How to use tile function in Keras?
- github issue: AttributeError: 'NoneType' object has no attribute '_inbound_nodes'
- Stackoverflow: Using Subtract layer in Keras
Here is the solution:
# instantiate a Keras tensor ... as per document https://keras.io/layers/core/
keras_tensor_input = keras.layers.Input( shape=[num_letters], dtype='float32' )
print("keras_tensor_input = ", keras_tensor_input)
# keras_tensor_input = Tensor("input_1:0", shape=(?, 3), dtype=float32)
# https://stackoverflow.com/questions/53865471/using-subtract-layer-in-keras
keras_tensor_neg_1 = keras.layers.Lambda(lambda x: -keras.backend.ones_like(x) )(keras_tensor_input)
print("keras_tensor_neg_1 = ", keras_tensor_neg_1)
# keras_tensor_neg_1 = Tensor("lambda_1/Neg:0", shape=(?, 3), dtype=float32)
# note batch size, just like keras_tensor_input
# https://stackoverflow.com/questions/53250533/how-to-use-tile-function-in-keras
keras_tensor_neg_1_tiled = keras.layers.Lambda(lambda x: keras.backend.tile(x, (1, n-1)))(keras_tensor_neg_1)
print("keras_tensor_neg_1_tiled = ", keras_tensor_neg_1_tiled)
# keras_tensor_neg_1_tiled = Tensor("Tile_2:0", shape=(12,), dtype=float32)
# note batch size, just like keras_tensor_input
# concatenate the input from the generator and the padding
keras_tensor_concat = keras.layers.Concatenate()( inputs = [keras_tensor_input, keras_tensor_neg_1_tiled] )
print("keras_tensor_concat = ", keras_tensor_concat)
# keras_tensor_concat = Tensor("concatenate_1/concat:0", shape=(?, 15), dtype=float32)
my_model = keras.models.Model( inputs=keras_tensor_input, output=keras_tensor_concat)
# dummy optimizer, loss, and metric so I can compile and test the model
my_model.compile(optimizer='rmsprop',loss='categorical_crossentropy', metrics=['accuracy'])
# a batch of size 1, for a tensor of shape =[3]
d1 = numpy.array([[1, 2, 3]])
out = my_model.predict( [ d1 ] )
print(out)
# [[ 1. 2. 3. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1.]]
来源：https://stackoverflow.com/questions/58158620/concatenate-input-tensor-with-multiple-of-neg-1-tensors