Hadoop gen1 vs Hadoop gen2

前端 未结 9 1253
南笙
南笙 2021-02-10 09:22

I am a bit confused about place of tasktracker in Hadoop-2.x.

Daemons in Hadoop-1.x are namenode, datanode, jobtracker, taskracker and secondaryna

9条回答
  •  面向向阳花
    2021-02-10 09:59

    What I get after reading above link is

    YARN handle the shortcomes of classic MR by splitting the functionality of Job tracker

    functionality of JobTracker in 1.x i.e resource management and job scheduling/monitoring are divided into separate daemons. - global ResourceManager (RM) and per-application ApplicationMaster (AM)

    ResourceManager - run at NameNode i.e master side

    • it DISTRIBUTE RESOURCES among all appl

      it has 2 main components: Scheduler and ApplicationsManager.

    • Scheduler is pure scheduler
    • ApplicationsManager is responsible for accepting job-submissions

    NodeManager - run at DataNode i.e slave side

    • is the per-machine framework agent
    • it is responsible for containers, monitoring their resource usage (cpu, memory, disk, network) and reporting the same to the ResourceManager/Scheduler.

    Central ResourceManager and Node specific Manager together is called YARN

提交回复
热议问题