I managed to remove all horizontale lines/rules in my chart using this:
scales: { xAxes: [{ gridLines: { display: false }
I found a way to remove this line. It's actually called the border of the axis and there's an option for it, see "Grid Line Configuration":
border
scales: { yAxes: [{ gridLines: { drawBorder: false, } }] }