How to get the schema definition from a dataframe in PySpark?

前端 未结 4 2046
萌比男神i
萌比男神i 2021-02-12 14:25

In PySpark it you can define a schema and read data sources with this pre-defined schema, e. g.:

Schema = StructType([ Str         


        
4条回答
  •  小鲜肉
    小鲜肉 (楼主)
    2021-02-12 14:29

    If you are looking for a DDL string from PySpark:

    df: DataFrame = spark.read.load('LOCATION')
    schema_json = df.schema.json()
    ddl = spark.sparkContext._jvm.org.apache.spark.sql.types.DataType.fromJson(schema_json).toDDL()
    

提交回复
热议问题