[YARN-9173] FairShare calculation broken for large values after YARN-8833

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.0
• Fix Version/s: 3.0.4, 3.1.2, 3.3.0, 3.2.1
• Component/s: fairscheduler
• Hadoop Flags: Reviewed
• Assignee: Wilfred Spiegelenburg
• Reporter: Wilfred Spiegelenburg
• Created: 03/Jan/19 11:22
• Updated: 26/Feb/20 05:29
• Resolved: 08/Jan/19 06:28

Description

After the fix for the infinite loop in YARN-8833 we now get the wrong values back for fairshare calculations under certain circumstances. The current implementation works when the total resource is smaller than Integer.MAXVALUE.

When the total resource goes above that value the number of iterations is not enough to converge to the correct value.

The new test testResourceUsedWithWeightToResourceRatio() only checks that the calculation does not hang but does not check the outcome of the calculation.

Issue Links

• is caused by: YARN-8833 Avoid potential integer overflow when computing fair shares
• is duplicated by: YARN-7560 Resourcemanager hangs when resourceUsedWithWeightToResourceRatio return a overflow value