Read range of files in pySpark

Question

I need to read contiguous files in pySpark. The following works for me.

from pyspark.sql import SQLContext    
file = \"events.parquet/exportDay=2015090[1-7]\"
         


        

Answer 1

Use curly braces.

file = "events.parquet/exportDay=201509{08,09,10,11,12,13,14}"

Here's a similar question on stack overflow: Pyspark select subset of files using regex glob. They suggest either using curly braces, OR performing multiple reads and then unioning the objects (whether they are RDDs or data frames or whatever, there should be some way).