Does ignore option of Pyspark DataFrameWriter jdbc function ignore entire transaction or just offending rows?

后端 未结 1 1466
春和景丽
春和景丽 2021-01-21 23:41

The Pyspark DataFrameWriter class has a jdbc function for writing a dataframe to sql. This function has an --ignore option that the documentation says

1条回答
  •  孤街浪徒
    2021-01-22 00:16

    mode("ingore") is just NOOP if table (or another sink) already exists and writing modes cannot be combined. If you're looking for something like INSERT IGNORE or INSERT INTO ... WHERE NOT EXISTS ... you'll have to do it manually, for example with mapPartitions.

    0 讨论(0)
提交回复
热议问题