ValueError: Series lengths must match to compare when matching dates in Pandas

Question

I apologize in advance for asking such a basic question but I am stumped.

This is a very simple, dummy example. I\'m having some issue matching dates in Pandas and I

Answer 1

As mentioned by DSM, some_date is a series and not a value. When you use boolean masking, and checking if value of a column is equal to some variable or not, we have to make sure that the variable is a value, not a container. One possible way of solving the problem is mentioned by DSM, there is also another way of solving your problem.

df[(df['ID']==some_id) & (df['Date'] == some_date.values[0])]

We have just replaced the some_date with some_date.values[0]. some_date.values returns an array with one element. We are interested in the value in the container, not the container, so we index it by [0] to get the value.

Answer 2

You say:

some_date = df.iloc[1:2]['Date']  # gives 2016-01-01

but that's not what it gives. It gives a Series with one element, not simply a value -- when you use [1:2] as your slice, you don't get a single element, but a container with one element:

>>> some_date
1   2016-01-01
Name: Date, dtype: datetime64[ns]

Instead, do

>>> some_date = df.iloc[1]['Date']
>>> some_date
Timestamp('2016-01-01 00:00:00')

after which

>>> df[(df['ID']==some_id) & (df['Date'] == some_date)] 
   ID       Date
0   1 2016-01-01

(Note that there are more efficient patterns if you have a lot of some_id and some_date values to look up, but that's a separate issue.)