YARN常用参数

我只是一个虾纸丫 提交于 2020-05-05 15:43:59

 

name value description
yarn.nodemanager.resource.memory-mb -1 Amount of physical memory, in MB, that can be allocated for containers. If set to -1 and yarn.nodemanager.resource.detect-hardware-capabilities is true, it is automatically calculated(in case of Windows and Linux). In other cases, the default is 8192MB.
yarn.scheduler.maximum-allocation-mb 8192 The maximum allocation for every container request at the RM, in MBs. Memory requests higher than this will throw a InvalidResourceRequestException.
yarn.scheduler.maximum-allocation-vcores 4 The maximum allocation for every container request at the RM, in terms of virtual CPU cores. Requests higher than this will throw a InvalidResourceRequestException.

yarn.log-aggregation-enable

false

Whether to enable log aggregation. Log aggregation collects each container's logs and moves these logs onto a file-system, for e.g. HDFS, after the application completes. Users can configure the "yarn.nodemanager.remote-app-log-dir" and "yarn.nodemanager.remote-app-log-dir-suffix" properties to determine where these logs are moved to. Users can access the logs via the Application Timeline Server.

yarn.log-aggregation.retain-seconds

-1

How long to keep aggregation logs before deleting them. -1 disables. Be careful set this too small and you will spam the name node.

yarn.log.server.url

 

URL for log aggregation server

yarn.nodemanager.remote-app-log-dir

/tmp/logs

Where to aggregate logs to.

 

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!