How to create a DataFrame from a text file in Spark

Question

I have a text file on HDFS and I want to convert it to a Data Frame in Spark.

I am using the Spark Context to load the file and then try to generate individual columns f

Answer 1

A txt File with PIPE (|) delimited file can be read as :

---

df = spark.read.option("sep", "|").option("header", "true").csv("s3://bucket_name/folder_path/file_name.txt")