Spark ExecutorLostFailure

Question

I\'m trying to run spark 1.5 on mesos in cluster mode. I\'m able to launch the dispatcher and to run the spark-submit. But when I do so, the spark driver fails with the followin

Answer 1

Your executors might be getting lost because of many different reasons, but the information you're getting (and showing) is not enough to understand why.

Even if I have no experience with Mesos in cluster mode, it seems to me that what you show as the executor logs are somehow incomplete: if you could get their complete logs you will see they are very helpful to determine the cause of such failures. I took a look at:

http://mesos.apache.org/documentation/latest/configuration/

and you should get the logs you're looking for from either their stderr (maybe you're just showing their stdout?). You could also try to use the parameter --log_dir=VALUE to dump their logs and understand better the situation.