Directly Convert PySpark RDD to Arrow Table

Question

I\'m mapping my PySpark RDD Partitions, and running a function across all partitions. query.rdd.mapPartitions(proc).collect()

Inside the function, I need the