在Spark 2.x当中,实际上SQLContext和HiveContext是过时的,相反是采用SparkSession对象的sql函数来操作SQL语句的。使用这个函数执行SQL语句前需要先调用DataFrame的createOrReplaceTempView注册一个临时表,所以关键是先要将RDD转换成DataFrame。实际上,在Spark中实际声明了
type DataFrame = Dataset[Row]
所以,DataFrame是Dataset[Row]的别名。RDD是提供面向低层次的API,而DataFrame/Dataset提供面向高层次的API(适合于SQL等面向结构化数据的场合)。
下面提供一些Spark SQL程序的例子。
例子一:SparkSQLExam.scala
https://weibo.com/ttarticle/p/show?id=2309404442671010676834
https://weibo.com/ttarticle/p/show?id=2309404442670872264752
https://weibo.com/ttarticle/p/show?id=2309404442670792573026
https://weibo.com/ttarticle/p/show?id=2309404442670654160901
https://weibo.com/ttarticle/p/show?id=2309404442670570275015
https://weibo.com/ttarticle/p/show?id=2309404442670490583375
https://weibo.com/ttarticle/p/show?id=2309404442670415085683
https://weibo.com/ttarticle/p/show?id=2309404442670343782592
https://weibo.com/ttarticle/p/show?id=2309404442670255702271
https://weibo.com/ttarticle/p/show?id=2309404442670092124353
https://weibo.com/ttarticle/p/show?id=2309404442669865631756
https://weibo.com/ttarticle/p/show?id=2309404442669735608538
https://weibo.com/ttarticle/p/show?id=2309404442669660110981
https://weibo.com/ttarticle/p/show?id=2309404442669576224878
https://weibo.com/ttarticle/p/show?id=2309404442669492339023
https://weibo.com/ttarticle/p/show?id=2309404442669412647079
https://weibo.com/ttarticle/p/show?id=2309404442669328760845
https://weibo.com/ttarticle/p/show?id=2309404442669240680680
https://weibo.com/ttarticle/p/show?id=2309404442669160988819
https://weibo.com/ttarticle/p/show?id=2309404442668997411013
https://weibo.com/ttarticle/p/show?id=2309404442668926107773
https://weibo.com/ttarticle/p/show?id=2309404442668846416031
https://weibo.com/ttarticle/p/show?id=2309404442668762529937
https://weibo.com/ttarticle/p/show?id=2309404442668687032415
https://weibo.com/ttarticle/p/show?id=2309404442668594757798
https://weibo.com/ttarticle/p/show?id=2309404442668515066019
https://weibo.com/ttarticle/p/show?id=2309404442667982389545
https://weibo.com/ttarticle/p/show?id=2309404442667898503260
https://weibo.com/ttarticle/p/show?id=2309404442667823005760
https://weibo.com/ttarticle/p/show?id=2309404442667726536890
https://weibo.com/ttarticle/p/show?id=2309404442667651039353
https://weibo.com/ttarticle/p/show?id=2309404442667575542013
https://weibo.com/ttarticle/p/show?id=2309404442667495850136
https://weibo.com/ttarticle/p/show?id=2309404442667416158521
https://weibo.com/ttarticle/p/show?id=2309404442667332272260
https://weibo.com/ttarticle/p/show?id=2309404442667256774837
https://weibo.com/ttarticle/p/show?id=2309404442667177083134
https://weibo.com/ttarticle/p/show?id=2309404442667097391315
https://weibo.com/ttarticle/p/show?id=2309404442667021893672
https://weibo.com/ttarticle/p/show?id=2309404442666946396571
https://weibo.com/ttarticle/p/show?id=2309404442666866704683
https://weibo.com/ttarticle/p/show?id=2309404442666766041207
https://weibo.com/ttarticle/p/show?id=2309404442666686349314
https://weibo.com/ttarticle/p/show?id=2309404442666589880657
https://weibo.com/ttarticle/p/show?id=2309404442666480828433
https://weibo.com/ttarticle/p/show?id=2309404442666090758162
https://weibo.com/ttarticle/p/show?id=2309404442666011066424
https://weibo.com/ttarticle/p/show?id=2309404442665692299525
https://weibo.com/ttarticle/p/show?id=2309404442665612607503
https://weibo.com/ttarticle/p/show?id=2309404442665516138682
https://weibo.com/ttarticle/p/show?id=2309404442665436446975
https://weibo.com/ttarticle/p/show?id=2309404442665222537485
https://weibo.com/ttarticle/p/show?id=2309404442665142845621
https://weibo.com/ttarticle/p/show?id=2309404442664979267913
https://weibo.com/ttarticle/p/show?id=2309404442664899576015
https://weibo.com/ttarticle/p/show?id=2309404442664819884131
https://weibo.com/ttarticle/p/show?id=2309404442664723415061
https://weibo.com/ttarticle/p/show?id=2309404442664647917761
https://weibo.com/ttarticle/p/show?id=2309404442664559837406
https://weibo.com/ttarticle/p/show?id=2309404442664316567769
https://weibo.com/ttarticle/p/show?id=2309404442664241070305
https://weibo.com/ttarticle/p/show?id=2309404442663481901255
https://weibo.com/ttarticle/p/show?id=2309404442663356072121
https://weibo.com/ttarticle/p/show?id=2309404442663150551055
https://weibo.com/ttarticle/p/show?id=2309404442663020527721
https://weibo.com/ttarticle/p/show?id=2309404442662961807615
https://weibo.com/ttarticle/p/show?id=2309404442662882115734
https://weibo.com/ttarticle/p/show?id=2309404442662798229652
https://weibo.com/ttarticle/p/show?id=2309404442662710149356
https://weibo.com/ttarticle/p/show?id=2309404442662622068910
https://weibo.com/ttarticle/p/show?id=2309404442662538182868