Google Cloud Storage bucket.get_blob to verified file path returns None

问题

I am able to verify the existence of finished_json_path in the bucket_name, but I get finished_json_blob value of None, when running this code... Any insights really appreciated!

    bucket_name = mdata_list[0]
    org_repo = mdata_list[3]
    pull_number = mdata_list[4]
    job_name = mdata_list[5]
    build_number = mdata_list[6]
    prlogs_pull_dir = bucket_name + "/pr-logs/pull"
    prlogs_directory_dir = bucket_name + "/pr_logs/directory"
    finished_json_path = prlogs_pull_dir + "/" + org_repo + "/" + pull_number + "/" + job_name + "/" + build_number + "/" + "finished.json"
    events_json_url = prlogs_pull_dir + "/" + org_repo + "/" + pull_number + "/" + job_name + "/" + build_number + "artifacts/build-resources/events.json"



    storage_client = storage.Client()
    bucket = storage_client.get_bucket(bucket_name)
    finished_json_blob = bucket.get_blob(finished_json_path)
    finished_json = finished_json_blob.download_as_string()

I have also replaced bucket.get_blob with bucket.blob, with some improvement, but still crashes out on .download_as_string. Not sure why I'm getting a pubsub message from this object when it's created but then not able to find it.

google.api_core.exceptions.NotFound: 404 GET https://storage.googleapis.com/download/storage/v1/b/origin-ci-test/o/origin-ci-test%2Fpr-logs%2Fpull%2Fopenshift_release%2F12691%2Frehearse-12691-pull-ci-operator-framework-operator-marketplace-release-4.6-okd-images%2F1326467646144647168%2Ffinished.json?alt=media: No such object: origin-ci-test/origin-ci-test/pr-logs/pull/openshift_release/12691/rehearse-12691-pull-ci-operator-framework-operator-marketplace-release-4.6-okd-images/1326467646144647168/finished.json: ('Request failed with status code', 404, 'Expected one of', <HTTPStatus.OK: 200>, <HTTPStatus.PARTIAL_CONTENT: 206>)

(myenv) bash-5.0$ gsutil cat gs://origin-ci-test/origin-ci-test/pr-logs/pull/openshift_release/12691/rehearse-12691-pull-ci-operator-framework-operator-marketplace-release-4.6-okd-images/1326467646144647168/finished.json
CommandException: No URLs matched: gs://origin-ci-test/origin-ci-test/pr-logs/pull/openshift_release/12691/rehearse-12691-pull-ci-operator-framework-operator-marketplace-release-4.6-okd-images/1326467646144647168/finished.json

回答1:

On the error can se that the full path you are calling is: origin-ci-test/origin-ci-test/pr-logs/pull/openshift_release/12691/rehearse-12691-pull-ci-operator-framework-operator-marketplace-release-4.6-okd-images/1326467646144647168/finished.json which seems to be that the bucket is passed twice.

The issue is that when calling get_blob you are already on the bucket but you are passing it again as part of finished_json_path

Here I am attaching he code removing the additional part of the path.

bucket_name = mdata_list[0]
org_repo = mdata_list[3]
pull_number = mdata_list[4]
job_name = mdata_list[5]
build_number = mdata_list[6]

prlogs_pull_dir = "/pr-logs/pull"
prlogs_directory_dir = bucket_name + "/pr_logs/directory"

finished_json_path = prlogs_pull_dir + "/" + org_repo + "/" + pull_number + "/" + job_name + "/" + build_number + "/" + "finished.json"
events_json_url = prlogs_pull_dir + "/" + org_repo + "/" + pull_number + "/" + job_name + "/" + build_number + "artifacts/build-resources/events.json"

storage_client = storage.Client()
bucket = storage_client.get_bucket(bucket_name)
finished_json_blob = bucket.get_blob(finished_json_path)
finished_json = finished_json_blob.download_as_string()

来源：https://stackoverflow.com/questions/64805849/google-cloud-storage-bucket-get-blob-to-verified-file-path-returns-none

标签

python

google-cloud-platform

google-cloud-storage