Filtering and comparing dates with Pandas

Question

I would like to know how to filter different dates at all the different time levels, i.e. find dates by year, month, day, hour, minute and/or day. For example, how do I find all

Answer 1

If you set timestamp as index and dtype as datetime to get a DateTimeIndex, then you can use the following Partial String Indexing syntax:

df['2014'] # gets all 2014
df['2014-01'] # gets all Jan 2014
df['01-02-2014'] # gets all Jan 2, 2014

Answer 2

You can filter your dataframe via boolean indexing like so:

df.loc[df['timeStamp'].dt.year == 2014]
df.loc[df['timeStamp'].dt.month == 5]
df.loc[df['timeStamp'].dt.second == 4]
df.loc[df['timeStamp'] == '2014-01-02']
df.loc[pd.to_datetime(df['timeStamp'].dt.date) == '2014-01-02']

... and so on and so forth.

Answer 3

I would just create a string series, then use str.contains() with wildcards. That will give you whatever granularity you're looking for.

s = df['timeStamp'].map(lambda x: x.strftime('%Y-%m-%d %H:%M:%S'))

print(df[s.str.contains('2014-..-.. ..:..:..')])
print(df[s.str.contains('2014-..-02 ..:..:..')])
print(df[s.str.contains('....-02-.. ..:..:..')])
print(df[s.str.contains('....-..-.. 18:03:10')])

Output:

        timeStamp
0 2014-01-02 21:03:04
1 2014-02-02 21:03:05
        timeStamp
0 2014-01-02 21:03:04
1 2014-02-02 21:03:05
        timeStamp
1 2014-02-02 21:03:05
2 2016-02-04 18:03:10
        timeStamp
2 2016-02-04 18:03:10

I think this also solves your question about boolean indices:

print(s.str.contains('....-..-.. 18:03:10'))

Output:

0    False
1    False
2     True
Name: timeStamp, dtype: bool