arrow-python

Validate an ISO-8601 datetime string in Python?

*爱你&永不变心* 提交于 2020-01-22 16:17:07
问题 I want to write a function that takes a string and returns True if it is a valid ISO-8601 datetime--precise to microseconds, including a timezone offset-- False otherwise. I have found other questions that provide different ways of parsing datetime strings, but I want to return True in the case of ISO-8601 format only. Parsing doesn't help me unless I can get it to throw an error for formats that don't match ISO-8601. (I am using the nice arrow library elsewhere in my code. A solution that

Parse date and time from string with time zone using Arrow

ぐ巨炮叔叔 提交于 2019-12-05 17:02:46
问题 I have import arrow s = '2015/12/1 19:00:00' tz = 'Asia/Hong_Kong' How can I parse this with Arrow such that I get an Arrow object with the time zone tz ? The following defaults to UTC time. In [30]: arrow.get(s, 'YYYY/M/D HH:mm:ss') Out[30]: <Arrow [2015-12-01T19:00:00+00:00]> I know the .to function but that converts a time zone and but doesn't allow me to change to time zone. 回答1: Try this: arrow.get(s, 'YYYY/M/D HH:mm:ss').replace(tzinfo=dateutil.tz.gettz(tz)) 回答2: I'm not qualified yet

How to read partitioned parquet files from S3 using pyarrow in python

*爱你&永不变心* 提交于 2019-11-28 20:22:50
问题 I looking for ways to read data from multiple partitioned directories from s3 using python. data_folder/serial_number=1/cur_date=20-12-2012/abcdsd0324324.snappy.parquet data_folder/serial_number=2/cur_date=27-12-2012/asdsdfsd0324324.snappy.parquet pyarrow's ParquetDataset module has the capabilty to read from partitions. So I have tried the following code : >>> import pandas as pd >>> import pyarrow.parquet as pq >>> import s3fs >>> a = "s3://my_bucker/path/to/data_folder/" >>> dataset = pq