Keep original data points when padding a signal with pandas

Question

Consider the following test data set:

testdf = pandas.DataFrame({\'t\': [datetime(2015, 1, 1, 10,  0),
                                 datetime(2015, 1, 1, 11,          


        

Answer 1

Generate an index with the missing timestamps and create a dataframe with NaN values. Then combine it with the combine_first method and fill in the NaN values:

idx = pandas.date_range(datetime(2015, 1, 1, 10, 0), datetime(2015, 1, 1, 12, 0), freq='30min')
df = pandas.DataFrame(numpy.nan, index=idx, columns=['val'])

testdf.set_index('t', inplace=True)
testdf.combine_first(df).fillna(method='ffill')

The documentation of the combine_first method reads:

Combine two DataFrame objects and default to non-null values in frame calling the method. Result index columns will be the union of the respective indexes and columns

The ffill method of the fillna method does the following (source):

ffill: propagate last valid observation forward to next valid backfill