google-cloud-ml-engine

How to package vocabulary file for Cloud ML Engine

限于喜欢 提交于 2020-02-25 04:47:50
问题 I have a .txt file which contains a different label on each line. I use this file to create a label index lookup file, for example: label_index = tf.contrib.lookup.index_table_from_file(vocabulary_file = 'labels.txt' I am wondering how I should package the vocabulary file with my cloud ml-engine? The packaging suggestions are explicit in how to set up the .py files but I am not entirely sure where I should put relevant .txt files. Should they just be hosted in a storage bucket (ie. gs://)

tensorflow Found more than one graph event per run

一个人想着一个人 提交于 2020-01-29 07:11:05
问题 I am loading a tensorboard for my ml engine experiment that is running in local mode and got the following warning: "Found more than one graph event per run, or there was a metagraph containing a graph_def, as well as one or more graph events. Overwriting the graph with the newest event. W0825 19:26:12.435613 Reloader event_accumulator.py:311] Found more than one metagraph event per run. Overwriting the metagraph with the newest event." Originally, I suspected that this was because I had not

Google Cloud ML Engine - Job failed due to an internal error . Can't execute a job

白昼怎懂夜的黑 提交于 2019-12-30 11:39:09
问题 This is a ml-job I previously trained successfully . But when I tried it today it's not working . So after that I tried removing all the things is the bucket and start over . Still it's not working . Giving the following error . Internal error occurred. Please retry in a few minutes. If you still experience errors, contact Cloud ML. 来源: https://stackoverflow.com/questions/45609164/google-cloud-ml-engine-job-failed-due-to-an-internal-error-cant-execute-a-j

reading files in google cloud machine learning

被刻印的时光 ゝ 提交于 2019-12-29 05:35:11
问题 I tried to run tensorflow-wavenet on the google cloud ml-engine with gcloud ml-engine jobs submit training but the cloud job crashed when it was trying to read the json configuration file: with open(args.wavenet_params, 'r') as f: wavenet_params = json.load(f) arg.wavenet_params is simply a file path to a json file which I uploaded to the google cloud storage bucket. The file path looks like this: gs://BUCKET_NAME/FILE_PATH.json . I double-checked that the file path is correct and I'm sure

reading files in google cloud machine learning

∥☆過路亽.° 提交于 2019-12-29 05:34:06
问题 I tried to run tensorflow-wavenet on the google cloud ml-engine with gcloud ml-engine jobs submit training but the cloud job crashed when it was trying to read the json configuration file: with open(args.wavenet_params, 'r') as f: wavenet_params = json.load(f) arg.wavenet_params is simply a file path to a json file which I uploaded to the google cloud storage bucket. The file path looks like this: gs://BUCKET_NAME/FILE_PATH.json . I double-checked that the file path is correct and I'm sure

Cloud ML Engine batch predictions - How to simply match returned predictions with input data?

我是研究僧i 提交于 2019-12-24 07:36:38
问题 According to the ML Engine documentation, an instance key is required to match the returned predictions with the input data. For simplicity purposes, I would like to use a DNNClassifier but apparently canned estimators don't seem to support instance keys yet (only custom or tensorflow core estimators). So I looked at the Census code examples of Custom/TensorflowCore Estimators but they look quite complex for what I am trying to achieve. I would prefer using a similar approach as described in

ML Engine Experiment eval tf.summary.scalar not displaying in tensorboard

邮差的信 提交于 2019-12-22 00:35:37
问题 I am trying to output some summary scalars in an ML engine experiment at both train and eval time. tf.summary.scalar('loss', loss) is correctly outputting the summary scalars for both training and evaluation on the same plot in tensorboard. However, I am also trying to output other metrics at both train and eval time and they are only outputting at train time. The code immediately follows tf.summary.scalar('loss', loss) but does not appear to work. For example, the code as follows is only

Using Training TFRecords that are stored on Google Cloud

前提是你 提交于 2019-12-21 07:14:51
问题 My goal is to use training data (format: tfrecords) stored on Google Cloud storage when I run my Tensorflow Training App, locally. (Why locally? : I am testing before I turn it into a training package for Cloud ML) Based on this thread I shouldn't have to do anything since the underlying Tensorflow API's should be able to read a gs://(url) However thats not the case and the errors I see are of the format: 2017-06-06 15:38:55.589068: I tensorflow/core/platform/cloud/retrying_utils.cc:77] The

Loading pre-trained word2vec to initialise embedding_lookup in the Estimator model_fn

倖福魔咒の 提交于 2019-12-21 05:05:13
问题 I am solving a text classification problem. I defined my classifier using the Estimator class with my own model_fn . I would like to use Google's pre-trained word2vec embedding as initial values and then further optimise it for the task at hand. I saw this post: Using a pre-trained word embedding (word2vec or Glove) in TensorFlow which explains how to go about it in 'raw' TensorFlow code. However, I would really like to use the Estimator class. As an extension, I would like to then train this

Internal Error on Google Cloud ML

只谈情不闲聊 提交于 2019-12-19 11:52:25
问题 I am getting an error "Internal error occurred for the current attempt" while submitting a job on google cloud ML. Can anyone help me on this ??? 回答1: We're experiencing a bit of a capacity crunch making it a little hard to obtain VMs. You can try running in another region such as us-east1 for the time being. We are working on making sure the error message is more informative. Thanks for your patience. 来源: https://stackoverflow.com/questions/45609471/internal-error-on-google-cloud-ml