Creating a simple 1-row Spark DataFrame with Java API

Question

In Scala, I can create a single-row DataFrame from an in-memory string like so:

val stringAsList = List(\"buzz\")
val df = sqlContext.sparkContext.parallelize(js         


        

Answer 1

You can achieve this by creating List to Rdd and than create Schema which will contain column name.

There might be other ways as well, it's just one of them.

List stringAsList = new ArrayList();
        stringAsList.add("buzz");

JavaRDD rowRDD = sparkContext.parallelize(stringAsList).map((String row) -> {
                return RowFactory.create(row);
            });

StructType schema = DataTypes.createStructType(new StructField[] { DataTypes.createStructField("fizz", DataTypes.StringType, false) });

DataFrame df = sqlContext.createDataFrame(rowRDD, schema).toDF();
df.show();

//+----+
|fizz|
+----+
|buzz|