[YARN-9914] Use separate configs for free disk space checking for full and not-full disks

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 2.10.0, 3.0.4, 3.3.0, 2.8.6, 2.9.3, 3.2.2, 3.1.4
• Component/s: yarn
• Assignee: Jim Brennan
• Reporter: Jim Brennan
• Created: 18/Oct/19 16:33
• Updated: 09/Dec/19 20:09
• Resolved: 28/Oct/19 14:22

Description

YARN-3943 added separate configurations for the nodemanager health check disk utilization full disk check:

max-disk-utilization-per-disk-percentage - threshold for marking a good disk full

disk-utilization-watermark-low-per-disk-percentage - threshold for marking a full disk as not full.

On our clusters, we do not use these configs. We instead use min-free-space-per-disk-mb so we can specify the limit in mb instead of percent of utilization. We have observed the same oscillation behavior as described in YARN-3943 with this parameter. I would like to add an optional config to specify a separate threshold for marking a full disk as not full:

min-free-space-per-disk-mb - threshold at which a good disk is marked full

disk-free-space-per-disk-high-watermark-mb - threshold at which a full disk is marked good.

So for example, we could set min-free-space-per-disk-mb = 5GB, which would cause a disk to be marked full when free space goes below 5GB, and disk-free-space-per-disk-high-watermark-mb = 10GB to keep the disk in the full state until free space goes above 10GB.