PySpark install error

Question

I have followed instructions from various blogs posts including this, this, this and this to install pyspark on my laptop. However when I try to use pyspark from terminal or jup

Answer 1

UnknownHostException is

Thrown to indicate that the IP address of a host could not be determined.

and it is thrown at the bottom of your stack trace:

Caused by: java.net.UnknownHostException: linux-0he7: Name or service not known

Looking at your prompt shell linux-0he7 so I assume you're using local mode. This means that your /etc/hosts doesn't include linux-0he7.

Adding

127.0.0.1    linux-0he7

to /etc/hosts should resolve the problem.

You can also use spark.driver.bindAddress and spark.driver.host to use specific host IP for the driver.

Independent of the exception Hadoop 3.0.0 is not supported yet. I would recommend using 2.x for the time being.