How to transform keras model to tpu model

问题

I am trying to transform my Keras model in the Google cloud console into a TPU model. Unfortunatelly I am getting an error as shown below. My minimal example is the following:

import keras
from keras.models import Sequential
from keras.layers import Dense, Activation
import tensorflow as tf
import os
model = Sequential()
model.add(Dense(32, input_dim=784))
model.add(Dense(32))
model.add(Activation('relu'))
model.compile(optimizer='rmsprop', loss='mse')
tpu_model = tf.contrib.tpu.keras_to_tpu_model(
    model,
    strategy=tf.contrib.tpu.TPUDistributionStrategy(
         tf.contrib.cluster_resolver.TPUClusterResolver(TPU_WORKER)))

My output is:

Using TensorFlow backend.
Traceback (most recent call last):
     File "cloud_python4.py", line 11, in <module>
     tpu_model = tf.contrib.tpu.keras_to_tpu_model(AttributeError: module 'tensorflow.contrib.tpu' has no attribute 'keras_to_tpu_model'

The keras_to_tpu_model method seems experimental as indicated on the tensorflow website. Has it recently been removed? If so, how can I proceed to make use of TPUs to estimate my Keras model? If the keras_to_tpu_model method would be still available, why can I not invoke it?

回答1:

I am assuming you defined you TPU_WORKER as below

import os
TPU_WORKER = ‘grpc://’ + os.environ[‘COLAB_TPU_ADDR’]

Instead of converting your model to TPU, build a distribution strategy. This is the method by which the batch will be distributed to the eight TPUs and how the loss from each will be calculated.

resolver = tf.contrib.cluster_resolver.TPUClusterResolver(TPU_WORKER)
tf.contrib.distribute.initialize_tpu_system(resolver)
strategy = tf.contrib.distribute.TPUStrategy(resolver)

With the strategy build and compile your model. This should work quite nicely for regression.

with strategy.scope():
  model = Sequential() 
  model.add(Dense(32, input_dim=784))
  model.add(Dense(32))
  model.add(Activation('relu'))
  model.compile(optimizer='rmsprop', loss='mse')

回答2:

Import keras from tensorflow. This is because tf.contrib.tpu.keras_to_tpu_model( )' requires a tensorflow version Model, not the keras version.

For example, use from tensorflow.keras.layers import Dense, Activation instead. And so on.

来源：https://stackoverflow.com/questions/54521572/how-to-transform-keras-model-to-tpu-model