gsutil

问题 I am trying to create an internal app to upload files to google cloud. I don't want each individual user or this app to log in so I'm using a service account. I login into the service account and everything is ok, but when I try to upload it gives me this error: ServiceException: 401 Anonymous caller does not have storage.objects.list access to bucket As you can see I am logged in with a service account and my account and(neither service or personal) works 回答1: I had similar problem, and as

问题 I am trying to create an internal app to upload files to google cloud. I don't want each individual user or this app to log in so I'm using a service account. I login into the service account and everything is ok, but when I try to upload it gives me this error: ServiceException: 401 Anonymous caller does not have storage.objects.list access to bucket As you can see I am logged in with a service account and my account and(neither service or personal) works 回答1: I had similar problem, and as

问题 I'm using a service account to upload a file to Google Cloud Storage bucket that has versioning. I want to keep the service account privileges minimal, it only ever needs to upload files so I don't want to give it permission to delete files, but the upload fails (only after streaming everything!) saying it requires delete permission. Shouldn't it be creating a new version instead of deleting? Here's the command: cmd-that-streams | gsutil cp -v - gs://my-bucket/${FILE}

问题 I'm using a service account to upload a file to Google Cloud Storage bucket that has versioning. I want to keep the service account privileges minimal, it only ever needs to upload files so I don't want to give it permission to delete files, but the upload fails (only after streaming everything!) saying it requires delete permission. Shouldn't it be creating a new version instead of deleting? Here's the command: cmd-that-streams | gsutil cp -v - gs://my-bucket/${FILE}

问题 Is there a way to list all public links for all the objects stored into a Google Cloud Storage bucket (or a directory in a bucket) using Cloud SDK's gsutil or gcloud ? Something like: $ gsutil ls --public-link gs://my-bucket/a-directory 回答1: Public links for publicly visible objects are predictable. They just match this pattern: https://storage.googleapis.com/BUCKET_NAME/OBJECT_NAME . gsutil doesn't have a command to print URLs for objects in a bucket, but it can just list objects. You could

问题 Is there a way to list all public links for all the objects stored into a Google Cloud Storage bucket (or a directory in a bucket) using Cloud SDK's gsutil or gcloud ? Something like: $ gsutil ls --public-link gs://my-bucket/a-directory 回答1: Public links for publicly visible objects are predictable. They just match this pattern: https://storage.googleapis.com/BUCKET_NAME/OBJECT_NAME . gsutil doesn't have a command to print URLs for objects in a bucket, but it can just list objects. You could

问题 I am trying to execute the following line gsutil -m rsync s3://input gs://output in python. When running this line in the shell terminal it works fine. However, I am trying to run this in a python script by using the following line. subprocess.Popen(["gsutil", "-m", "rsync", "s3://input", "gs://output"]) However it just hangs forever. It outputs the following: Building synchronization state... Starting synchronization... The bash command successfully prints: Building synchronization state...

问题 I am trying to execute the following line gsutil -m rsync s3://input gs://output in python. When running this line in the shell terminal it works fine. However, I am trying to run this in a python script by using the following line. subprocess.Popen(["gsutil", "-m", "rsync", "s3://input", "gs://output"]) However it just hangs forever. It outputs the following: Building synchronization state... Starting synchronization... The bash command successfully prints: Building synchronization state...

问题 I am trying to setup a schedule job/process in cloud to load csv data into Bigquery from google buckets using a python script. I have manage to get hold off the python code to do this but not sure where do I need to save this code so that this task could be completed as an automated process rather than running the gsutil commands manualy. 回答1: Reliable Task Scheduling on Google Compute Engine | Solutions | Google Cloud Platform, the 1st link in Google on "google cloud schedule a cron job",

问题 I've figured that this command lists paths to all files: gsutil ls "gs://bucket/foldername/*.csv" This command imports a file to BQ and autodetects schema: bq load --autodetect --source_format=CSV dataset.tableName gs://bucket/foldername/something.csv Now I need to make it work together to import all files to respective tables in BQ. If table exists, then replace it. Could you give me a hand? 回答1: First, create a file with all the list with all the folders you want to load into BigQuery: