问题
In the gcloud documentation for google bigquery, it states that authentication can be determined from from_service_account_json.
I've tried the following
from gcloud import bigquery
client = bigquery.Client.from_service_account_json('/Library/gcloud_api_credentials.json')
The json file looks like the following (Note: Scrambled credentials so these are now fake).
{"type": "service_account",
"project_id": "example_project",
"private_key_id": "c7e371776ab6e2dsfafdsaff97edf9377178c8",
"private_key": "-----BEGIN PRIVATE KEY-----\nXXXXXXXXAgEAAoIBAQDBIAaPzRVOT12J\nfPKzJ/tgVTJWHWiO29C/OpjRbd4WUFNEfX3mdShYoVo9+F7u3QSlCxA4SgZmro0s\n/yQKyhCrpcmrmWJyr5VqqIvqScJVJzBvLHCto/JoydCSzYQJz89vltZ11NZE1n+W\ncvF2ygX9oM/dyUK/1eVsJOIoj4qj2jpRuQhxgh0ag2HVYlmAfnP9wfTdLeoV0e/z\n0WtBNesCrekh/ooldbxcDa6KvLJpBKu6BJg0nYWtnB0Q2hEZl1msNDs9lsMIWI/4\nRwz4iEv67HDTDaTerF89sFsPW5dyybnspxuNXlNSjtdABpHTU4mm1bwgRSEomuyZ\nptMp+Ck/AgMBAAECggEAYXyDE2/Yw3D4rIBwGtOh5tzazfGGaCa51u4DWiR1qRnf\nDoGyXfTh6ut4HDQV58mVoKJXC7MCTC7sgLxCO1gI1jyX8pE4yt+rMu3lwtx8xnND\n1bq/HkfWA+Nr787iSfmmopOnqk9H4fPM7+sRzH7qSz8z7qa7lQ6qAs8TgpoCx/Zt\nQsfAUDBFHP2x2msngZVkW2rZiCJyiWDE62KI7uxlUXlnqkC9j6vXL+n/SBmVGYSn\n2crQ0CONdBXvk753E4e3AhhA7VUcQorhEnYGFcsI2L9s1mK2SOGnakAukyZpoqQG\n//UmCX/lSu4SpfKi+NFedtQLfZqpfU6tOhS90OzYwQKBgQD1v1FdJ05X92RzRlSb\nQUUqWNKC3OwSlFAEZnBrpY8xSlK8alLXQRQWhBjdugtnfRfvPokILoxtZcgC6rUY\ntyauXPdDbv8+U4+YtcRSf3ZqNQmIajwt/pS/aj9CpY+ZL62codVmXOnpjiaIEZLk\nECJC947oYpxBb6k5OIrecZHuzwKBgQDJLq5H4i3JNpmIHiC/xgDcL9ROrPJBZhaS\fdafsdafddsffdsf/+\nCzA/tpS/dn0i5bf4E/SFWJdFHXk/rIooEGypFNw9e2Sw5rElI3B9DYKkXpHWDLvS\ne0vHm7s6kQKBgQDAH0ZY8XXKR9r8VCupBSoxpSdOr7DQBatZBU02m0SLYHO2YdlJ\n99Lgl31pOx3XVnDz1YwEHYAjtsLbpOrX27gd8QhOa+SYsXM+DDzj0kqdVcGSKJvL\nyZA45bN3Q0A0npW4Mr1HPbSRp1RwZGzWZ9ahTDEpdI58ifNejNnkdh2E+QKBgQCQ\nkSDfuuJX8wpEzBGQfUJk92HC33h+ElQp8MOyl+2qrzTt5KQIkJ+7gbG1F2lQ/Qhc\nqaIDTPTozJds1YfpLARc/fdsfddfd/ovcaSayzV8dEGeXDKU7GrVds5\niwPqRHk3HRJwJHYrT9onoqu8XdDCVggp6ju8RmY2YQKBgQC+r2aGJTOE5BNEwoZa\nT5q5lda+ADFm9JQj9zcF6OI4UFQqJaW0KVlPdr2pH5h6PoRtyVsZ5ZbDSjbZ2+Jp\nT1fW54HoK2S+faXWEmHPIYbRGSSDRJ/Kphu28D4C8Jo8YJKn4rwd8dnWmYC/73cx\naqUiJ7TdFLUYNwl/tg/XGR3PTQ==\n-----END PRIVATE KEY-----\n",
"client_email": "example_project@appspot.gserviceaccount.com",
"client_id": "123422110336236066294",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://accounts.google.com/o/oauth2/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/example-project%40appspot.gserviceaccount.com"
}
I get the below error:
Traceback (most recent call last):
File "/Applications/PyCharm.app/Contents/helpers/pydev/pydevd.py", line 2217, in <module>
globals = debugger.run(setup['file'], None, None)
File "/Applications/PyCharm.app/Contents/helpers/pydev/pydevd.py", line 1643, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Shippy/API/google_big_query_api.py", line 69, in <module>
try2()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Shippy/API/google_big_query_api.py", line 62, in try2
client = bigquery.Client.from_service_account_json('/Library/gcloud_api_credentials.json')
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/gcloud-0.10.1-py2.7.egg/gcloud/client.py", line 59, in from_service_account_json
credentials = get_for_service_account_json(json_credentials_path)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/gcloud-0.10.1-py2.7.egg/gcloud/credentials.py", line 128, in get_for_service_account_json
json_credentials_path, scopes=scope)
File "build/bdist.macosx-10.6-x86_64/egg/oauth2client/service_account.py", line 195, in from_json_keyfile_name
File "build/bdist.macosx-10.6-x86_64/egg/oauth2client/service_account.py", line 171, in _from_parsed_json_keyfile
File "build/bdist.macosx-10.6-x86_64/egg/oauth2client/service_account.py", line 108, in __init__
File "build/bdist.macosx-10.6-x86_64/egg/oauth2client/util.py", line 163, in scopes_to_string
TypeError
I think I may have incorrectly created the json credentials file, but I'm not sure how that is suppose to be made.
I've also tried to download the json file from google, but now get the following error:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-3-79ea8cfc1391> in <module>()
----> 1 client = bigquery.Client.from_service_account_json('/Users/Chris/Desktop/Shippy-abc820f485e5.json')
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/gcloud-0.10.1-py2.7.egg/gcloud/client.pyc in from_service_account_json(cls, json_credentials_path, *args, **kwargs)
57 if 'credentials' in kwargs:
58 raise TypeError('credentials must not be in keyword arguments')
---> 59 credentials = get_for_service_account_json(json_credentials_path)
60 kwargs['credentials'] = credentials
61 return cls(*args, **kwargs)
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/gcloud-0.10.1-py2.7.egg/gcloud/credentials.pyc in get_for_service_account_json(json_credentials_path, scope)
126 """
127 return ServiceAccountCredentials.from_json_keyfile_name(
--> 128 json_credentials_path, scopes=scope)
129
130
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/oauth2client-2.0.0.post1-py2.7.egg/oauth2client/service_account.pyc in from_json_keyfile_name(cls, filename, scopes)
193 with open(filename, 'r') as file_obj:
194 client_credentials = json.load(file_obj)
--> 195 return cls._from_parsed_json_keyfile(client_credentials, scopes)
196
197 @classmethod
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/oauth2client-2.0.0.post1-py2.7.egg/oauth2client/service_account.pyc in _from_parsed_json_keyfile(cls, keyfile_dict, scopes)
169 credentials = cls(service_account_email, signer, scopes=scopes,
170 private_key_id=private_key_id,
--> 171 client_id=client_id)
172 credentials._private_key_pkcs8_pem = private_key_pkcs8_pem
173 return credentials
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/oauth2client-2.0.0.post1-py2.7.egg/oauth2client/service_account.pyc in __init__(self, service_account_email, signer, scopes, private_key_id, client_id, user_agent, **kwargs)
106 self._service_account_email = service_account_email
107 self._signer = signer
--> 108 self._scopes = util.scopes_to_string(scopes)
109 self._private_key_id = private_key_id
110 self.client_id = client_id
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/oauth2client-2.0.0.post1-py2.7.egg/oauth2client/util.pyc in scopes_to_string(scopes)
161 return scopes
162 else:
--> 163 return ' '.join(scopes)
164
165
TypeError:
I've also tried downloading the p12 file
bigquery.Client.from_service_account_p12('chris@curalate.com','/Users/Chris/Desktop/Shippy-e824e0396911.p12')
I then receive the below error:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-9-e71586550945> in <module>()
----> 1 client = bigquery.Client.from_service_account_p12('chris@curalate.com','/Users/Chris/Desktop/Shippy-e824e0396911.p12')
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/gcloud-0.10.1-py2.7.egg/gcloud/client.pyc in from_service_account_p12(cls, client_email, private_key_path, *args, **kwargs)
92 raise TypeError('credentials must not be in keyword arguments')
93 credentials = get_for_service_account_p12(client_email,
---> 94 private_key_path)
95 kwargs['credentials'] = credentials
96 return cls(*args, **kwargs)
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/gcloud-0.10.1-py2.7.egg/gcloud/credentials.pyc in get_for_service_account_p12(client_email, private_key_path, scope)
157 """
158 return ServiceAccountCredentials.from_p12_keyfile(
--> 159 client_email, private_key_path, scopes=scope)
160
161
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/oauth2client-2.0.0.post1-py2.7.egg/oauth2client/service_account.pyc in from_p12_keyfile(cls, service_account_email, filename, private_key_password, scopes)
276 return cls._from_p12_keyfile_contents(
277 service_account_email, private_key_pkcs12,
--> 278 private_key_password=private_key_password, scopes=scopes)
279
280 @classmethod
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/oauth2client-2.0.0.post1-py2.7.egg/oauth2client/service_account.pyc in _from_p12_keyfile_contents(cls, service_account_email, private_key_pkcs12, private_key_password, scopes)
245 signer = crypt.Signer.from_string(private_key_pkcs12,
246 private_key_password)
--> 247 credentials = cls(service_account_email, signer, scopes=scopes)
248 credentials._private_key_pkcs12 = private_key_pkcs12
249 credentials._private_key_password = private_key_password
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/oauth2client-2.0.0.post1-py2.7.egg/oauth2client/service_account.pyc in __init__(self, service_account_email, signer, scopes, private_key_id, client_id, user_agent, **kwargs)
106 self._service_account_email = service_account_email
107 self._signer = signer
--> 108 self._scopes = util.scopes_to_string(scopes)
109 self._private_key_id = private_key_id
110 self.client_id = client_id
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/oauth2client-2.0.0.post1-py2.7.egg/oauth2client/util.pyc in scopes_to_string(scopes)
161 return scopes
162 else:
--> 163 return ' '.join(scopes)
164
165
TypeError:
回答1:
From https://googlecloudplatform.github.io/gcloud-python/latest/gcloud-api.html:
json_credentials_path (string) – The path to a private key file (this file was given to you when you created the service account). This file must contain a JSON object with a private key and other credentials information (downloaded from the Google APIs console).
It looks like the file should have been downloaded when you created the service account. Let's look at service account creation. Hiding in the storage docs we find a nice step-by-step for creating a JSON credential file.
From https://cloud.google.com/storage/docs/authentication#generating-a-private-key:
To generate a private key in JSON or PKCS12 format:
- Open the list of existing credentials in the Google Cloud Platform Console.
- Click New credentials and select Service account.
- In the Create service account window, select the Key type, either JSON or P12.
- Click Create.
- A New public/private key pair window is displayed and the private key for the Key type you selected is downloaded automatically. If you selected a P12 key, the private key's password ("notasecret") is displayed.
- Click Close.
来源：https://stackoverflow.com/questions/35529331/how-to-authenticate-with-gcloud-big-query-using-a-json-credentials-file