Insert missing weekdays in pandas dataframe and fill them with NaN

Question

I am trying to insert missing weekdays in a time series dataframe such has

import pandas as pd
from pandas.tseries.offsets import *
df = pd.DataFrame([['2016-09-30', 10, 2020], ['2016-10-03', 20, 2424], ['2016-10-05', 5, 232]], columns=['date', 'price', 'vol']).set_index('date')
df['date'] = pd.to_datetime(df['date'])
df = df.set_index('date')

data looks like this :

Out[300]: 
            price   vol
date                   
2016-09-30     10  2020
2016-10-03     20  2424
2016-10-05      5   232

I can create a series of week days easily with pd.date_range()

pd.date_range('2016-09-30', '2016-10-05', freq=BDay())
Out[301]: DatetimeIndex(['2016-09-30', '2016-10-03', '2016-10-04', '2016-10-05'], dtype='datetime64[ns]', freq='B')

based on that DateTimeIndex I would like to add missing dates in my dfand fill column values with NaN so I get:

Out[300]: 
            price   vol
date                   
2016-09-30     10  2020
2016-10-03     20  2424
2016-10-04     NaN  NaN
2016-10-05      5   232

is there an easy way to do this? Thanks!

Answer 1

You can use reindex:

df.index = pd.to_datetime(df.index)

df.reindex(pd.date_range('2016-09-30', '2016-10-05', freq=BDay()))
Out: 
            price     vol
2016-09-30   10.0  2020.0
2016-10-03   20.0  2424.0
2016-10-04    NaN     NaN
2016-10-05    5.0   232.0

Answer 2

Alternatively, you can use pandas.DataFrame.resample(), specifying 'B' for Business Day with no need to specify beginning or end date sequence as along as the dataframe maintains a datetime index

df = df.resample('B').sum()

#             price     vol
# date                     
# 2016-09-30   10.0  2020.0
# 2016-10-03   20.0  2424.0
# 2016-10-04    NaN     NaN
# 2016-10-05    5.0   232.0