Problem with data transfer from Cloud Build container to Google Compute Engine instance

问题

Currently I'm using Cloud Build to produce some artifacts that I need to deploy to GCE instance. I've tried to use gcloud builder for this purpose with the following args:

- name: 'gcr.io/cloud-builders/gcloud' 
  args: ['compute', 'scp', '--zone=<zone_id>', '<local_path>', '<google compute engine instance name>:<instance_path>']

and build fails with the following error:

ERROR: (gcloud.compute.scp) Could not SSH into the instance.  It is 
possible that your SSH key has not propagated to the instance yet. Try 
running this command again.  If you still cannot connect, verify that 
the firewall and instance are set to accept ssh traffic.

I've already opened port 22 on my instance but that haven't helped me.
Could you guys help me to solve this problem?
What points I need to check/fix in my build definition?
May be you can give me an advice which builder instead of gcloud I can use to deliver my data from Cloud Build container to the GCE instance?

回答1:

A few things to try:

1.Make sure you can ssh normally this way.

Troubleshooting SSH if step one fails.

2.Try to change the SSH target from 'instancename' to 'username@instance' in order to indicate the name of the user inside the VM, eg

username@InstanceName

回答2:

You must find a way to generate and locace the SSH Key Files for the builder to connect to the GCE Instance:

• google_compute
• google_compute.pub
• google_compute_known_hosts

They are identical to the ones you use to directly connect to the instance from your Cloud Shell or from your Local Computer, but this time the connection has to be done by the builder it self.

Make that files interactively like explained in SSH Key Generation to the identity path of builder (test it by cd ~ && pwd, usually: /builder/home/.ssh).

After a connection has been made then copy these files to Google Cloud Storage via gsutil. This step is need to be done one time only.

steps:

- name: 'gcr.io/cloud-builders/gsutil'
  args: ['cp', '-rP', '${_BUIKDER_HOME}', 'gs://${_BUCKET_NAME}/builder/']

substitutions:
  _BUCKET_NAME: <bucket_name>
  _BUIKDER_HOME: <builder_home>

timeout: "60s"

You might take those key files to your workspace. If you prefer as it then they will need to be remain stay in the storage.

The purpose of this placement is that they will be used to reconnect to the instance because each time the builder is started it will be configured back to the default stage so the files will no more exist.

Once the key files are ready, then you can do the scp transfer like below:

steps:

- name: 'gcr.io/cloud-builders/gsutil'
  args: ['cp', '-rP',  'gs://${_BUCKET_NAME}/builder/.ssh'], '_${_BUILDER_HOME}']

- name: 'gcr.io/cloud-builders/gcloud' 
  args: ['compute', 'scp', '--recurse', '--zone', '${_ZONE}', '${_LOCAL_PATH}', '${_USER_NAME}@${_INSTANCE_NAME}:${INSTANCE_PATH}']

substitutions:
  _ZONE: <zone>
  _USER_NAME: <user_name>
  _LOCAL_PATH: <local_path>
  _BUCKET_NAME: <bucket_name>
  _BUILDER_HOME: : <builder_home>
  _INSTANCE_NAME: <instance_name>
  _INSTANCE_PATH: <instance_path>

timeout: "60s"

Note: Use the flag of '--recurse' to copy a directory or none to copy a file only.

来源：https://stackoverflow.com/questions/52204216/problem-with-data-transfer-from-cloud-build-container-to-google-compute-engine-i