ggplot2 - How to plot a CI only in the predicted line

Question

I used a GNM (Generalised Non-Linear Model) to make a prediction and stored all the data in a data frame. I also stored a pred_up and pred_down var