Plot pandas dataframe containing NaNs

Question

I have GPS data of ice speed from three different GPS receivers. The data are in a pandas dataframe with an index of julian day (incremental from the start of 2009).

Answer 1

Given that you want to draw a straight line between the points where you do have data, you can get Pandas to fill in the gaps via interpolation, and then plot:

.interpolate(method='linear').plot()

Answer 2

The reason your not seeing anything is because the default plot style is only a line. But the line gets interupted at NaN's so only multiple consequtive values will be plotted. And the latter doesnt happen in your case. You need to change the style of plotting, which depends on what you want to see.

For starters, try adding:

.plot(marker='o')

That should make all data points appear as circles. It easily gets cluttered so adjusting markersize, edgecolor etc might be usefull. Im not fully adjusted to how Pandas is using matplotlib so i often switch to matplotlib myself if plots get more complicated, eg:

plt.plot(df.R2.index.to_pydatetime(), df.R2, 'o-')

Answer 3

Here is another way:

nan_columns = []
nan_values = []

for column in dataset.columns:
    nan_columns.append(column)
    nan_values.append(dataset[column].isnull().sum())

fig, ax = plt.subplots(figsize=(30,10))
plt.bar(nan_columns, nan_values)

Answer 4

I found even if the df was indexed as DateTime the same issues occurred. One solution to ensure all data points are respected, with no gaps in between lines, is to plot each df column separately and dropping the NaNs.

    for col in df.columns:
        plot_data = df[col].dropna()
        ax.plot(plot_data.index.values, plot_data.values, label=col)