Details
-
Improvement
-
Status: Patch Available
-
Major
-
Resolution: Unresolved
-
None
-
None
Description
When a yarn cluster shares the same hosts as the underlying HDFS cluster, we have yarn.resourcemanager.nodes.include-path point to the same file or symlink of the dfs.hosts file used by HDFS to make admin easier.
If we want to set up a yarn cluster to run on the same hosts of several HDFS clusters combined, it means yarn.resourcemanager.nodes.include-path should be able to point to a list of files each of which belongs to one HDFS cluster.
Backward compatibility, it seems ok to continue to reuse yarn.resourcemanager.nodes.include-path as long as it can still take a single file.