Map-Reduce Logs on Hive-Tez

社会主义新天地 提交于 2019-12-04 09:11:51

The log you posted is DAG execution log. The DAG consists of Map 1,Map 3,Map 4,Map 7 mappers vertices and reducers: Reducer 2,Reducer 5,Reducer 6

Map 1: -/- - this means that the vertex is not initialized, the number of mappers are not calculated yet.

Map 4: 0(+7)/118 - this means that totally there are 118 mappers, 7 of them are running in parallel, 0 completed yet, 118-7=111 are pending.

Reducer 2: 0/15 - this means that totally there are 15 reducers, 0 of them are running, 0 of them are completed (15 reducers pending).

Negative figures (there are no such in your example) = number of failed or killed mappers or reducers

Qubole has explanation about Tez log: https://docs.qubole.com/en/latest/user-guide/hive/using-hive-on-tez/hive-tez-tuning.html#understanding-log-pane

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!