问题
I am trying to use Google Datalab to read in a file in ipython notebook, the basic pd.read_csv() as I can't find the path of the file. I have it locally and also uploaded it to google cloud storage in a bucket.
I ran the following commands to understand where I am
os.getcwd()
gives '/content/myemail@gmail.com'
os.listdir('/content/myemail@gmail.com')
gives ['.git', '.gitignore', 'datalab', 'Hello World.ipynb', '.ipynb_checkpoints']
回答1:
The following reads the contents of the object into a string variable called text
:
%%storage read --object "gs://path/to/data.csv" --variable text
Then
from cStringIO import StringIO
mydata = pd.read_csv(StringIO(text))
mydata.head()
Hopefully Pandas will support "gs://"
URLs (as it does for s3://
currently to allow reading directly from Google Cloud storage.
I have found the following docs really helpful:
https://github.com/GoogleCloudPlatform/datalab/tree/master/content/datalab/tutorials
Hope that helps (just getting started with Datalab too, so maybe someone will have a cleaner method soon).
回答2:
You can also run BigQuery queries directly against CSV files in Cloud Storage by creating a FederatedTable wrapper object. That is described here:
https://github.com/GoogleCloudPlatform/datalab/blob/master/content/datalab/tutorials/BigQuery/Using%20External%20Tables%20from%20BigQuery.ipynb
来源:https://stackoverflow.com/questions/34713864/reading-in-a-file-with-google-datalab