Affects Version/s: 2.7.1, 2.6.2
Fix Version/s: None
Currently, the default value of yarn.scheduler.maximum-allocation-vcores is 32, according to yarn-default.xml.
However, in YarnConfiguration.java, we specify the default to be 4.
The default in the code looks correct to me. Actually I feel that the default value should be the same as yarn.nodemanager.resource.cpu-vcores (whose default is 8) ---if we have 8 cores for scheduling, there's few reason we only allow the maximum of 4...
The Cloudera's article on Tuning the Cluster for MapReduce v2 (YARN) also suggests that "the maximum value ( yarn.scheduler.maximum-allocation-vcores) is usually equal to yarn.nodemanager.resource.cpu-vcores..."
At the very least, we should fix yarn-default.xml. The error is pretty bad. A simple search on the Internet shows some ppl are confused by this error, for example,
(but seriously, I think we should have an automatic defaults with the min as 1 and the max equal to the number of cores on the machine...