In my case this line was culprit,
GridLayoutManager gridLayoutManager = new GridLayoutManager(this, SPAN_COUNT, GridLayoutManager.VERTICAL, true);
Specifically , last parameter which is setting reverseLayout
attribute to true. Make it false
and RecyclerView doesn't scroll to bottom of screen.