reading a file in hdfs from pyspark

Question

I\'m trying to read a file in my hdfs. Here\'s a showing of my hadoop file structure.

hduser@GVM:/usr/local/spark/bin$ hadoop fs -ls -R /
drwxr-xr-x   - hduser s         


        

Answer 1

Since you don't provide authority URI should look like this:

hdfs:///inputFiles/CountOfMonteCristo/BookText.txt

otherwise inputFiles is interpreted as a hostname. With correct configuration you shouldn't need scheme at all an use:

/inputFiles/CountOfMonteCristo/BookText.txt

instead.