[YARN-8394] Improve data locality documentation for Capacity Scheduler

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.2.0, 3.1.1, 3.0.4
• Hadoop Flags: Reviewed
• Assignee: Weiwei Yang
• Reporter: Weiwei Yang
• Created: 05/Jun/18 07:43
• Updated: 01/Nov/18 05:07
• Resolved: 13/Jun/18 05:59

Description

YARN-6344 introduces a new parameter yarn.scheduler.capacity.rack-locality-additional-delay in capacity-scheduler.xml, we need to add some documentation in CapacityScheduler.md accordingly.

Moreover, we are seeing more and more clusters are separating storage and computation where file system is always remote, in such cases we need to introduce how to compromise data locality in CS otherwise MR jobs are suffering.

Issue Links

• is related to: YARN-8965 Revisit delay scheduling for cloud environment