google storage bucket file link publicly accessible even though not public?

Question

I was playing around with google bucket. The bucket is not public. The files are also not public.

After i upload the .csv file. I click on it and it shows the file

Answer 1

The following assumes the bucket name is xtest and the object name is test.txt.

That long complicated URL contains a signature that provides permissions to access the object.

If the URL looks very complicated and does not look like this, then it probably has a signature as part of the URL.

http://xtest.storage.googleapis.com/test.txt

OR

http://storage.googleapis.com/xtest/test.txt

If the URL does not contain a signature that allows anyone to access the bucket object, then the next steps are to figure out what permissions have been applied that allow anonymous access.

Figure out what permissions are applied to the bucket and object.

I prefer to use the CLI gsutil so that I have precise JSON describing all permissions.

There are two methods to grant access to buckets and objects. Bucket ACLs and Bucket IAM Policies.

PART 1 - Bucket ACLs

Get the Bucket ACL.

gsutil acl get gs://xtest

This will return a JSON response. If the bucket acl contains either of the following entries, your bucket is exposed.

[
  {
    "entity": "allUsers",
    "role": "READER"
  },
  {
    "entity": "allAuthenticatedUsers",
    "role": "READER"
  }
]

Remove public permissions.

The allUsers entity allows anyone the permissions specified by role. The allAuthenticatedUsers entity allows anyone with a Google Account the permissions specified by role.

This command will remove allUsers from the bucket ACL.

gsutil acl ch -d allUsers gs:/xtest

This command will remove allAuthenticatedUsers from the bucket ACL.

gsutil acl ch -d allAuthenticatedUsers gs:/xtest

When changing ACLs on a bucket or file, it can take about a minute to take effect.

Repeat the process for the object:

gsutil acl get gs://xtest/test.txt

Using similar commands to remove any public ACLs:

gsutil acl ch -d allUsers gs://xtest/test.txt

gsutil acl ch -d allAuthenticatedUsers gs://xtest/test.txt

Repeat verifying that public ACLs have been removed.

gsutil acl get gs://xtest

gsutil acl get gs://xtest/test.txt

Part 2 - Bucket IAM Policies

Get the Bucket IAM Policy.

gsutil iam get gs://xtest

This will return a JSON response. If the bucket IAM policy contains either of the following entries, your bucket is exposed.

{
  "bindings": [
    {
      "members": [
        "allUsers"
      ],
      "role": "roles/storage.legacyBucketReader"
    },
    {
      "members": [
        "allAuthenticatedUsers"
      ],
      "role": "roles/storage.objectViewer"
    }
  ],
  "etag": "CBM="
}

Remove public permissions.

The allUsers entity allows anyone the permissions specified by role. The allAuthenticatedUsers entity allows anyone with a Google Account the permissions specified by role.

This command will remove allUsers from the bucket IAM policy.

gsutil iam ch -d allUsers gs://xtest

This command will remove allAuthenticatedUsers from the bucket IAM policy.

gsutil iam ch -d allAuthenticatedUsers gs://xtest

Repeat the process for the object:

gsutil iam get gs://xtest/test.txt

Using similar commands to remove any public object IAM policies:

gsutil iam ch -d allUsers gs://xtest/test.txt

gsutil iam ch -d allAuthenticatedUsers gs://xtest/test.txt

Repeat verifying that public IAM policies have been removed.

gsutil iam get gs://xtest

gsutil iam get gs://xtest/test.txt