Hortonworks Hadoop NN and RM heap stuck overloaded at high utilization, but no applications running? (java.io.IOException: No space left on device)

Question

Had some recent Spark jobs initiated from a Hadoop (HDP-3.1.0.0) client node that raised some

Exception in thread "main" org.apache.hadoop.fs.FSErr