Apache Spark vs Apache Spark 2 [closed]

无人久伴 提交于 2019-12-10 01:24:09

问题


What are the improvements Apache Spark2 brings compared to Apache Spark?

  1. From architecture perspective
  2. From application point of view
  3. or more

回答1:


Apache Spark 2.0.0 APIs have stayed largely similar to 1.X, Spark 2.0.0 does have API breaking changes

Apache Spark 2.0.0 is the first release on the 2.x line. The major updates are API usability, SQL 2003 support, performance improvements, structured streaming, R UDF support, as well as operational improvements.

New in spark 2:

  • The biggest change that I can see is that DataSet and DataFrame APIs will be merged.
  • The latest and greatest from Spark will be a whole lot efficient as compared to predecessors. Spark 2.0 is going to focus on a combination of Parquet and caching to achieve even better throughput.
  • Structured streaming is another big thing!
  • It will be the first version that will focus on ETL. Successive versions will add more operators and libraries for ETL

You can go through the Spark release 2.0.0 where updates in following points are explained:

  • API Stability
  • Core and Spark SQL
  • MLlib
  • SparkR
  • Streaming
  • Dependency, Packaging, and Operations
  • Removals, Behavior Changes and Deprecations
  • Known Issues



回答2:


There is not much difference with respect to architecture as the nutshell is still DAG and RDD , which is the most important part of it !

Though Spark 2.0 is much more optimized and has DataSet Api which gives much more powerful to the hands of developers. So I would say the architecture is same it is just the Spark 2.0 provides much optimized and has a rich set of Api !

These are the main things that are provided by Apache Spark 2.0:

  • The biggest change that I can see is that DataSet and DataFrame APIs will be merged.
  • The latest and greatest from Spark will be a whole lot efficient as compared to predecessors. Spark 2.0 is going to focus on a combination of Parquet and caching to achieve even better throughput.
  • Structured streaming is another big thing!
  • It will be the first version that will focus on ETL. Successive versions will add more operators and libraries for ETL

For more information please take a lok here : https://www.quora.com/What-are-special-features-and-advantages-of-Apache-Spark-2-0-over-earlier-versions



来源:https://stackoverflow.com/questions/40168779/apache-spark-vs-apache-spark-2

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!