I am getting an error every so often when I try to populate my RecyclerView but the error seems to happening internally in the StaggeredGridLayoutManager.
I think this problem is triggered by https://code.google.com/p/android/issues/detail?id=77846.
Can you replace notifyDataSetChanged calls with specific notify events (notifyItemRemoved/Added) and see if you can reproduce it?