How to execute .sql file in spark using python

Question

from pyspark import SparkConf, SparkContext
from pyspark.sql import SQLContext

conf = SparkConf().setAppName(\"Test\").set(\"spark.driver.memory\", \"1g\")
sc = Spa         


        

Answer 1

SqlContext.sql expects a valid SQL query not a path to the file. Try this:

with open("/home/ubuntu/workload/queryXX.sql") as fr:
   query = fr.read()
results = sqlContext.sql(query)