matplotlib set ylabel padding to zero

Question

I have a plot

and I want to move the ylabel closer to the Y axis. I achieve this by axes.set_ylabel(\'neuron\', labelpad=-20) but -20