how to read and write to the same file in spark using parquet?
问题 I am trying to read from a parquet file in spark, do a union with another rdd and then write the result into the same file I have read from (basically overwrite), this throws the following error: couldnt write parquet to file: An error occurred while calling o102.parquet. : org.apache.spark.sql.catalyst.errors.package$TreeNodeException: execute, tree: TungstenExchange hashpartitioning(billID#42,200), None +- Union :- Scan ParquetRelation[units#35,price#36,priceSold#37,orderingTime#38,itemID