PredictionIO Engine

Question

Facing the error due to iteration in engine for data due to which stackoverflow exception is coming as follows:

ERROR org.apache.spark.executor.         


        

Answer 1

Got a solution for the error:

1.Simply reduce the numIterations parameter for algorithm in engine.json file in your prediction engine.

or If this does'nt work go with another solution below.

2.Add checkpointing, which prevents the recursion used by the codebase from creating an overflow. First, create a new directory to store the checkpoints. Then, have your SparkContext use that directory for checkpointing. Here is the example in Python:

sc.setCheckpointDir('checkpoint/') You may also need to add checkpointing to the ALS as well, but I haven't been able to determine whether that makes a difference. To add a checkpoint there (probably not necessary), just do:

ALS.checkpointInterval = 2