For work under umbrella of HADOOP-8468, user can enable nodegroup layer between node and rack in some situations. We should document it after YARN-18 and YARN-19 is figured out.
Document for enabling node group layer in HDFS
Configurable Hierarchical Topology for YARN
4-layer topology (with NodeGroup layer) implementation of Container Assignment and Task Scheduling (for YARN)
documentation for configuring HVE: dfs.block.replicator.classname should be org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyWithNodeGroup
When YARN part of work is done, the whole feature is completed and we should document it well.
It's kind of dumb at this point waiting years for some other piece that may never materialize. Clearly people are using the feature and it is complete enough that some documentation should exist. Plus HDFS-6261 exists too.
Agree with Allen Wittenauer. We can have a document before YARN code get checked in. Given HDFS-6261 is almost there, let's mark this JIRA as duplicated. We can have a separated one for YARN document when patch is there.