Export GCP Datastore and import to a different GCP Project

问题

I'm trying to copy data from DataStore in one GCP project to another - the process doesn't seem to give any errors, but also doesn't populate my target datastore.

In my source project I create a bucket and export my datastore in to it as follows :

gcloud datastore export gs://ctb_datatransfer

I then switch to my destination project and run

gcloud datastore import gs://ctb_datatransfer/2020-03-01T18:41:37_32872/2020-03-01T18:41:37_32872.overall_export_metadata

Neither of these steps seem to indicate any errors, but my destination datastore remains empty after the import has completed.

Any help is gratefully received as I don't really have much of a clue about what is happening - I believe that I have granted the correct permissions on the bucket ( and would expect to see some sort of permissions failure if I hadn't I guess )

The output from the export step looks like this

    done: true
metadata:
  "@type": type.googleapis.com/google.datastore.admin.v1.ExportEntitiesMetadata
  common:
    endTime: "2020-03-01T18:41:50.349159Z"
    operationType: EXPORT_ENTITIES
    startTime: "2020-03-01T18:41:37.188012Z"
    state: SUCCESSFUL
  entityFilter: {}
  outputUrlPrefix: gs://ctb_datatransfer/2020-03-01T18:41:37_32872
  progressBytes:
    workCompleted: "8848"
    workEstimated: "6480"
  progressEntities:
    workCompleted: "40"
    workEstimated: "40"
name: projects/chattyappsdata-staging/operations/ASAzMTEwMzAzMTMJGnRsdWFmZWQHEjJ3LXVlLXNib2otbmltZGEQCigS
response:
  "@type": type.googleapis.com/google.datastore.admin.v1.ExportEntitiesResponse

while the output from the import step looks like this

    done: true
metadata:
  "@type": type.googleapis.com/google.datastore.admin.v1.ImportEntitiesMetadata
  common:
    endTime: "2020-03-01T18:46:25.251008Z"
    operationType: IMPORT_ENTITIES
    startTime: "2020-03-01T18:45:44.029056Z"
    state: SUCCESSFUL
  entityFilter: {}
  inputUrl: gs://ctb_datatransfer/2020-03-01T18:41:37_32872/2020-03-01T18:41:37_32872.overall_export_metadata
  progressBytes:
    workCompleted: "8110"
    workEstimated: "8848"
  progressEntities:
    workCompleted: "40"
    workEstimated: "40"
name: projects/chattytoolbox/operations/AiAwNTEwNDAxMTMJGnRsdWFmZWQHEjJ3LXVlLXNib2otbmltZGEQCigS
response:
  "@type": type.googleapis.com/google.protobuf.Empty

回答1:

As stated by OP in the comments, the whole process ended up working, it just did it hours later. This is a possible explanation:

Looking at the documentation about importing entities, I've found that the import is a long running operation (LRO) and thus it may take a substantial amount of time to complete. From the import output you shared, we can see that the operation did finish, but I believe it was missing some propagation, which got fixed by time.

Also, the import command (gcloud datastore import) it just starts the LRO. If you specify the argument --async the command will return immediately but if you don't, it will wait for a response. On this section, on the gcloud tab, it's stated that even if the import command gets cancelled the operation will continue, thus reinforcing the theory about the independence of the command, the import and the propagation.

来源：https://stackoverflow.com/questions/60479107/export-gcp-datastore-and-import-to-a-different-gcp-project

标签

google-cloud-platform

google-cloud-datastore