How to import data to Dataset and retrain custom model in Google Cloud AutoML

问题

I'm a new developer with GCP and I learn about Google Cloud AutoML Custom Model. but I have 2 problems with AutoML Vision.

1. I can't import data from csv file in cloud storage to Dataset. I'm using C# to call RestAPI but its error 404. below is my code.

var uri = "https://automl.googleapis.com/v1beta1/projects/{project-id}/locations/us-central1/datasets/{dataset-id}:import";

        var request = (HttpWebRequest)WebRequest.Create(uri);
        request.Method = "POST";
        request.ContentType = "application/json";
        request.Headers.Add("Authorization", "Bearer " + _token);

        using (var streamWriter = new StreamWriter(request.GetRequestStream()))
        {
            string json = "{\"inputUris\":\"gs://{bucket-name}/Vehicles/csv/{csv-file-name}.csv\"}";
            Console.WriteLine(json);
            streamWriter.Write(json);
            streamWriter.Flush();
            streamWriter.Close();
        }

        try
        {
            var httpResponse = (HttpWebResponse)request.GetResponse();
            using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
            {
                var result = streamReader.ReadToEnd();
                Console.WriteLine(result);
            }
        }
        catch (Exception ex)
        {
            Console.WriteLine(ex.Message);
        }

2. How cant i Retrain Custom Model using C# or RestAPI

For example: the user uploads a new image of something with a label for that image. then create a csv file and upload to cloud storage. So I would like import Dataset using that csv file then retrain custom model only the image inside csv file (Add new image training to old model).

回答1:

In adition to @Awais answer, The correct call api is

https://automl.googleapis.com/v1beta1/projects/{id-project}/locations/us-central1/datasets/{id-dataset}:importData

and the correct json format payload for this function is

{
    "inputConfig": {
        "gcsSource": {
            "inputUris": [
                "gs://my-bucket-vcm/uploads/app/csv/19_03_2019_18_16_35.csv"
            ]
        }
    }
}

Source

回答2:

Answer of question 1: I suggest you to re-check the csv file, check this link Example:

gs://my-project-lcm/training-data/file1.txt,Sports,Basketball
gs://my-project-lcm/training-data/ubuntu.zip,Computers,Software,Operating_Systems,Linux,Ubuntu
file://news/documents/file2.txt,Sports,Baseball
"Miles Davis was an American jazz trumpeter, bandleader, and composer.",Arts_Entertainment,Music,Jazz
TRAIN,gs://my-project-lcm/training-data/astros.txt,Sports,Baseball
VALIDATE,gs://my-project-lcm/training-data/mariners.txt,Sports,Baseball
TEST,gs://my-project-lcm/training-data/cubs.txt,Sports,Baseball

Answer of question 2: I think that when you retrain your dataset (with new images), it will create a new model using all datset (with new images). If you look at the list of models, you will see that you will have 2 models and a dataset.

This is the curl to use if you want to import the dataset:

curl 
  -X POST 
  -H "Authorization: Bearer here-access-token" 
  -H "Content-Type: application/json" 
  https://automl.googleapis.com/v1beta1/projects/{id-project}/locations/us-central1/datasets/{id-dataset}:import \
  -d '{
    "inputUris": "gs://name-bucket-vcm/csv/file-csv.csv",
  }'

Here is python code:

import requests

url = "https://automl.googleapis.com/v1beta1/projects/{id-project}/locations/us-central1/datasets/{id-dataset}:import"

payload = "{"inputUris": "gs://bucket-vcm/csv/file-csv.csv"}"
headers = {
    'Content-Type': "application/json"
    }

response = requests.request("POST", url, data=payload, headers=headers)

print(response.text)

来源：https://stackoverflow.com/questions/52638410/how-to-import-data-to-dataset-and-retrain-custom-model-in-google-cloud-automl