How to extract a bz2 file in spark

问题

I have a csv file zipped in bz2 format, like unix/linux do we have any single line command to extrac/decompress the file file.csv.bz2 to file.csv in spark-scala?

回答1:

You can use built in function in SparkContext(sc), this worked for me

sc.textFile("file.csv.bz2").saveAsTextFile("file.csv")

来源：https://stackoverflow.com/questions/52981195/how-to-extract-a-bz2-file-in-spark