Rows missing from spark dataframes after using window and filter function

Question

I have a spark dataframe on which I want to do a stratified split. I referred to the answer given in the SO post https://stackoverflow.com/a/50476540/5743766 to do the split