When to use Kryo serialization in Spark?

后端 未结 3 1831
梦谈多话
梦谈多话 2021-02-19 13:17

I am already compressing RDDs using conf.set(\"spark.rdd.compress\",\"true\") and persist(MEMORY_AND_DISK_SER). Will using Kryo serialization make the

3条回答
  •  春和景丽
    2021-02-19 13:57

    Considering another point: kyro is faster than the default in serialization and deserialization, so it's better to use kyro. But the performance increase may be not as good as said, there are other points which will influence the program speed, like how you write your spark code, which lib you choose.

提交回复
热议问题