h2o starting on YARN not working

Question

When I start H2o on a cdh cluster I get the following error. I downloaded everything formt he wbesite and followed the tutorial. The command I ran was

hadoop j         


        

Answer 1

You should setup your default queue to have available resources to run 2nodes cluster.

See warnings:

1. WARNING: Job memory request (2.2 GB) exceeds queue available memory capacity (0.0 GB)

   • you ask 1GB per node (+overhead) but there is no available resources in the YARN queue

2. WARNING: Job virtual cores request (2) exceeds queue available virtual cores capacity (0)

   • you ask for 2 virtual cores but no cores are available in your default queue

Please check YARN documentation - for example setup of capacity scheduler and max available resources: https://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html

Answer 2

I made the following changes in Cloudera Manager yarn configuration

Setting                                     Value
yarn.scheduler.maximum-allocation-vcores    8 
yarn.nodemanager.resource.cpu-vcores        4
yarn.nodemanager.resource.cpu-vcores        4
yarn.scheduler.maximum-allocation-mb        16 GB