I updated the support library to 23.2.0. and changed the height of the recyclerView to wrap_content as explained in the blog here :
The RecyclerView widg
I found the bug.
In the layout file where the cardview is in, I had to modify the surrounding RelativeLayout's layout_height attribute to WRAP_CONTENT as well.
Interesting part is, I changed the layout_height of RecyclerView back to MATCH_PARENT and it still worked as desired.