google-cloud-tpu

问题 I am using TPU runtime in Google Colab, but having problems in reading files (not sure). I initialized TPU using: import tensorflow as tf import os import tensorflow_datasets as tfds resolver = tf.distribute.cluster_resolver.TPUClusterResolver(tpu='grpc://' + os.environ['COLAB_TPU_ADDR']) tf.config.experimental_connect_to_cluster(resolver) # This is the TPU initialization code that has to be at the beginning. tf.tpu.experimental.initialize_tpu_system(resolver) print("All devices: ", tf.config

来源： https://stackoverflow.com/questions/63302446/colabtpu-not-supporting-tf-2-3-0-tf-keras-layers-experimental-preprocessing

问题 I'm using Google colab TPU to train a simple Keras model. Removing the distributed strategy and running the same program on the CPU is much faster than TPU . How is that possible? import timeit import os import tensorflow as tf from sklearn.datasets import load_iris from sklearn.model_selection import train_test_split from tensorflow.keras.models import Sequential from tensorflow.keras.layers import Dense from tensorflow.keras.optimizers import Adam # Load Iris dataset x = load_iris().data y

问题 I'm using Google colab TPU to train a simple Keras model. Removing the distributed strategy and running the same program on the CPU is much faster than TPU . How is that possible? import timeit import os import tensorflow as tf from sklearn.datasets import load_iris from sklearn.model_selection import train_test_split from tensorflow.keras.models import Sequential from tensorflow.keras.layers import Dense from tensorflow.keras.optimizers import Adam # Load Iris dataset x = load_iris().data y

问题 We are trying to build an image segmentation deep learning model using Google Colab TPU. Our model is Mask R-CNN. TPU_WORKER = 'grpc://' + os.environ['COLAB_TPU_ADDR'] import tensorflow as tf tpu_model = tf.contrib.tpu.keras_to_tpu_model( model.keras_model, strategy=tf.contrib.tpu.TPUDistributionStrategy( tf.contrib.cluster_resolver.TPUClusterResolver(TPU_WORKER))) However I am running into issues while converting our Mask R-CNN model to TPU model as pasted below. ValueError: Layer <keras

问题 My model is too big to get a batch >64 with the normal v2 TPU devices. On the troubleshooting site it is mentioned that upcoming tensorflow versions will have bfloat16 support. Are the newly supported tf versions 1.9-1.12 capable to use bfloat16 now and if yes, is there a limited set of optimizers I can use? I did not find any further documentation on this but saw the usage of bfloat16 in the tensor2tensor model, so I guess there must be a way. Furthermore I read that TPU v3 supports bigger

问题 My model is too big to get a batch >64 with the normal v2 TPU devices. On the troubleshooting site it is mentioned that upcoming tensorflow versions will have bfloat16 support. Are the newly supported tf versions 1.9-1.12 capable to use bfloat16 now and if yes, is there a limited set of optimizers I can use? I did not find any further documentation on this but saw the usage of bfloat16 in the tensor2tensor model, so I guess there must be a way. Furthermore I read that TPU v3 supports bigger

问题 I'm using Talos and Google colab TPU to run hyperparameter tuning of a Keras model. Note that I'm using Tensorflow 1.15.0 and Keras 2.2.4-tf. import os import tensorflow as tf import talos as ta from tensorflow.keras.models import Sequential from tensorflow.keras.layers import Dense from tensorflow.keras.optimizers import Adam from sklearn.model_selection import train_test_split def iris_model(x_train, y_train, x_val, y_val, params): # Specify a distributed strategy to use TPU resolver = tf

问题 I'm using Talos and Google colab TPU to run hyperparameter tuning of a Keras model. Note that I'm using Tensorflow 1.15.0 and Keras 2.2.4-tf. import os import tensorflow as tf import talos as ta from tensorflow.keras.models import Sequential from tensorflow.keras.layers import Dense from tensorflow.keras.optimizers import Adam from sklearn.model_selection import train_test_split def iris_model(x_train, y_train, x_val, y_val, params): # Specify a distributed strategy to use TPU resolver = tf

问题 I'm trying to fine-tune a Huggingface transformers BERT model on TPU. It works in Colab but fails when I switch to a paid TPU on GCP. Jupyter notebook code is as follows: [1] model = transformers.TFBertModel.from_pretrained('bert-large-uncased-whole-word-masking-finetuned-squad') # works [2] cluster_resolver = tf.distribute.cluster_resolver.TPUClusterResolver( tpu='[My TPU]', zone='us-central1-a', project='[My Project]' ) tf.config.experimental_connect_to_cluster(cluster_resolver) tf.tpu