Why does upload_from_file Google Cloud Storage Function Throws timeout error?

问题

I created a function that works for me, It uploads a file to Google Cloud Storage.

The problem is when my friend tries to upload the same file to the same bucket using the same code from his local machine, he gets timeout error. His internet is very good and he should be able to upload the file with no problems within his connections.

Any idea why is this happening?

def upload_to_cloud(file_path):
    """
        saves a file in the google storage. As Google requires audio files greater than 60 seconds to be saved on cloud before processing
        It always saves in 'audio-files-bucket' (folder)
        Input:
            Path of file to be saved
        Output:
            URI of the saved file
    """
    print("Uploading to cloud...")
    client = storage.Client().from_service_account_json(KEY_PATH)
    bucket = client.get_bucket('audio-files-bucket')
    file_name = str(file_path).split('\\')[-1]
    print(file_name)
    blob = bucket.blob(file_name)
    f = open(file_path, 'rb')
    blob.upload_from_file(f)
    f.close()
    print("uploaded at: ", "gs://audio-files-bucket/{}".format(file_name))
    return "gs://audio-files-bucket/{}".format(file_name)

It throughs the timeout exception in upload_from_file(f) line.

We tried to use upload_from_filename function, but the same error still occurs.

回答1:

The problem is solved by reducing the chunk size of the blob. The code changed to be:

def upload_to_cloud(file_path):
    """
        saves a file in the google storage. As Google requires audio files greater than 60 seconds to be saved on cloud before processing
        It always saves in 'audio-files-bucket' (folder)
        Input:
            Path of file to be saved
        Output:
            URI of the saved file
    """
    print("Uploading to cloud...")
    client = storage.Client().from_service_account_json(KEY_PATH)
    bucket = client.get_bucket('audio-files-bucket')
    file_name = str(file_path).split('\\')[-1]
    print(file_name)
    blob = bucket.blob(file_name)

    ## For slow upload speed
    storage.blob._DEFAULT_CHUNKSIZE = 2097152 # 1024 * 1024 B * 2 = 2 MB
    storage.blob._MAX_MULTIPART_SIZE = 2097152 # 2 MB

    with open(file_path, 'rb') as f:
        blob.upload_from_file(f)
    print("uploaded at: ", "gs://audio-files-bucket/{}".format(file_name))
    return "gs://audio-files-bucket/{}".format(file_name)

来源：https://stackoverflow.com/questions/61001454/why-does-upload-from-file-google-cloud-storage-function-throws-timeout-error

标签

python

google-cloud-platform

google-cloud-storage