Spark 2.x - How to generate simple Explain/Execution Plan
问题 I am hoping to generate an explain/execution plan in Spark 2.2 with some actions on a dataframe. The goal here is to ensure that partition pruning is occurring as expected before I kick off the job and consume cluster resources. I tried a Spark documentation search and a SO search here but couldn't find a syntax that worked for my situation. Here is a simple example that works as expected: scala> List(1, 2, 3, 4).toDF.explain == Physical Plan == LocalTableScan [value#42] Here's an example