Multiple images input to the same CNN using Conv3d in keras

Question

I want to enter 8 images at the same time to the same CNN structure using conv3d. my CNN model is as following:

def build(sample, frame, height, width, chann         


        

Answer 1

** Edit: updated the link Here is a custom imagedatagenerator for 5D input to Conv3D nets. Hope it helps. Here is an example on how to use it:

from tweaked_ImageGenerator_v2 import ImageDataGenerator
datagen = ImageDataGenerator()
train_data=datagen.flow_from_directory('path/to/data', target_size=(x, y), batch_size=32, frames_per_step=4)

OR

You can build your own 5D tensor:

frames_folder = 'path/to/folder'
X_data = []
y_data = []
list_of_sent = os.listdir(frames_folder)
print (list_of_sent)
class_num = 0
time_steps = 0  
frames = []
for i in list_of_sent:
    classes_folder = str(frames_folder + '/' + i) #path to each class
    print (classes_folder)
    list_of_frames = os.listdir(classes_folder)
    time_steps= 0
    frames = []
    for filename in  sorted(list_of_frames):   
        if ( time_steps == 8 ):
            X_data.append(frames) #appending each tensor of 8 frames resized to 110,110
            y_data.append(class_num) #appending a class label to the set of 8 frames
            j = 0  
            frames = []
        else:
            time_steps+=1
            filename = cv2.imread(vid + '/' + filename)
            filename = cv2.resize(filename,(110, 110),interpolation=cv2.INTER_AREA)
            frames.append(filename)


    class_num+=1
X_data = np.array(X_data)
y_data = np.array(y_data)

For the snippet above, the folder structure must be like that:

    data/
        class0/
            img001.jpg
            img002.jpg
            ...
        class1/
            img001.jpg
            img002.jpg
            ...