How to view Random Forest statistics in Spark (scala)

Question

I have a RandomForestClassifierModel in Spark. Using .toDebugString() outputs the following

Tree 0 (weight 1.0):
  If (feature 0 in {1.0,2.0,3.0})
   If (feature         


        

Answer 1

One way I found useful yesterday was I could use spark.read.parquet() function to read the output from the model/data file. This way all information about a certain node could be retrieved as a whole dataframe.

`val modelPath = "some/path/to/your/model"
val dataPath = modelPath + "/data"    
val nodeData: DataFrame = spark.read.parquet(dataPath)
nodeData.show(500,false)
nodeData.printSchema()`

Then you can rebuild the tree with information. Hope it helps.