Spark fails to start in local mode when disconnected [Possible bug in handling IPv6 in Spark??]

Question

The problem is the same as described here Error when starting spark-shell local on Mac

... but I have failed to find a solution. I also used to get the malformed URI err

Answer 1

To those who are working with spark through sbt and having the same issue. Just add .set("spark.driver.host", "localhost") to your SparkConf() so initialisation of spark context will look like this:

val conf = 
    new SparkConf()
    .setAppName( "temp1" )
    .setMaster( "local" )
    .set( "spark.driver.host", "localhost" )

val sc = 
    SparkContext
    .getOrCreate( conf )

This initial configuration must be done before any other getOrCreate of SparkContext.