Read File spark, set field having specific value to null or “”

前端 未结 2 570
[愿得一人]
[愿得一人] 2021-01-17 04:15

I\'m Reading a text file delimited with | . There are some fields having value \\N . When read the file row by row to a data-frame, is there any way to make the field having

2条回答
  •  走了就别回头了
    2021-01-17 04:22

    "DataFrameNaFunctions" can be used for replace value "\N" in all columns with "":

    df.na.replace(df.columns.toSeq, Map("\\N" -> ""))
    

提交回复
热议问题