Federated learning : convert my own image dataset into tff simulation Clientdata

问题

here is the code of my federated learning test

from __future__ import absolute_import, division, print_function
import os
import collections
import warnings
from six.moves import range
import numpy as np
import six
import tensorflow as tf
import tensorflow_federated as tff
from tensorflow.keras.callbacks import EarlyStopping, ReduceLROnPlateau, ModelCheckpoint
import PIL


#pretrain

train_datagen1 = tf.keras.preprocessing.image.ImageDataGenerator(vertical_flip=True)
training_set1= train_datagen1.flow_from_directory('folder1/train',target_size=(200, 200), batch_size=32)



)

Now when I want to create sample_batch like the tutorial in the tensorflow federtaed for image classification

I write this line and it find this error

example_dataset = training_set1.create_tf_dataset_for_client(training_set1.client_ids[0])

the error

TypeError Traceback (most recent call last) in 1 training_set1.element_type_structure ----> 2 example_dataset = training_set1.create_tf_dataset_for_client(training_set1.client_ids[0])

TypeError: 'abstractproperty' object does not support indexing

Can you tell me how I must do to create dummy_batch in order to convert keras model into tff.learning.from_compiled_keras_model(model, dummy_batch)

回答1:

Thanks for your interest in TFF!

Generally, TFF is designed to ingest tf.data.Dataset objects, so the example above needs a little extra preprocessing.

Good news is, there is an existing tutorial showing an example of doing this. In the above, something like the following should work:

ds = tf.data.Dataset.from_generator(
    img_gen.flow_from_directory, args=[<your_directory>], 
    output_types=<your_types>, 
    output_shapes=<your_shapes>
)

Generally, one can think about the ClientData object as being a fancy dict mapping client ids to tf.data.Datasets. ClientData itself is an abstract class, and so can't be directly instantiated, and classmethods are provided to construct real instantiations of ClientData. One such classmethod that should work here would be tff.simulation.ClientData.from_clients_and_fn. Here, if you pass a list of client_ids and a function which returns the appropriate dataset when given a client id, you will have your hands on a fully functional ClientData.

I think here, an approach for defining the function you may use is to construct a Python dict which maps client IDs to tf.data.Dataset objects--you could then define a function which takes a client id, looks up the dataset in the dict, and returns the dataset.

Hopefully this helps!

来源：https://stackoverflow.com/questions/59741397/federated-learning-convert-my-own-image-dataset-into-tff-simulation-clientdata

标签

python

tensorflow

keras

federated

tensorflow-federated