How to convert List to JavaRDD

前端 未结 4 777
爱一瞬间的悲伤
爱一瞬间的悲伤 2021-02-11 15:54

We know that in spark there is a method rdd.collect which converts RDD to a list.

List f= rdd.collect();
String[] array = f.toArray(new String[f.si         


        
4条回答
  •  暗喜
    暗喜 (楼主)
    2021-02-11 16:48

    Adding to Sean Owen and others solutions

    You can use JavaSparkContext#parallelizePairs for List ofTuple

    List> pairs = new ArrayList<>();
    pairs.add(new Tuple2<>(0, 5));
    pairs.add(new Tuple2<>(1, 3));
    
    JavaSparkContext sc = new JavaSparkContext();
    
    JavaPairRDD rdd = sc.parallelizePairs(pairs);
    

提交回复
热议问题