Is there some rule of thumb or best practice regarding number of stages in Spark job? When do you consider breaking job in smaller pieces?
I found smaller jobs easier