Unable to generate UUIDs in Spark SQL

后端 未结 1 560
北恋
北恋 2021-01-25 08:39

below is the code block and the error recieved

> creating a temporary views 
sqlcontext.sql(\"\"\"CREATE TEMPORARY VIEW temp_pay         


        
相关标签:
1条回答
  • 2021-01-25 09:10

    Here is a Simple Example How you can generate timeuuid :

    import org.apache.spark.sql.SQLContext    
    val sqlcontext = new SQLContext(sc)
    
    import sqlcontext.implicits._
    
    //Import UUIDs that contains the method timeBased()
    import com.datastax.driver.core.utils.UUIDs
    
    //user define function timeUUID  which will retrun time based uuid      
    val timeUUID = udf(() => UUIDs.timeBased().toString)
    
    //sample query to test, you can change it to yours
    val df_newrecords = sqlcontext.sql("SELECT 1 as data UNION SELECT 2 as data").withColumn("time_uuid", timeUUID())
    
    //print all the rows
    df_newrecords.collect().foreach(println)
    

    Output :

    [1,9a81b3c0-170b-11e7-98bf-9bb55f3128dd]
    [2,9a831350-170b-11e7-98bf-9bb55f3128dd]
    

    Source : https://stackoverflow.com/a/37232099/2320144 https://docs.datastax.com/en/drivers/java/2.0/com/datastax/driver/core/utils/UUIDs.html#timeBased--

    0 讨论(0)
提交回复
热议问题