Exclude a particular area from geom_smooth fit automatically

Question

I am plotting different plots in my shiny app.

By using geom_smooth(), I am fitting a smoothing curve on a scatterplot.

I am plotting these plots