[YARN-1781] NM should allow users to specify max disk utilization for local disks

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.4.0
• Component/s: nodemanager
• Hadoop Flags: Reviewed
• Assignee: Varun Vasudev
• Reporter: Varun Vasudev
• Created: 04/Mar/14 18:16
• Updated: 29/Aug/14 17:31
• Resolved: 08/Mar/14 00:53

Description

This is related to YARN-257(it's probably a sub task?). Currently, the NM does not detect full disks and allows full disks to be used by containers leading to repeated failures. YARN-257 deals with graceful handling of full disks. This ticket is only about detection of full disks by the disk health checkers.

The NM should allow users to set a maximum disk utilization for local disks and mark disks as bad once they exceed that utilization. At the very least, the NM should at least detect full disks.

Issue Links

• breaks: YARN-2473 YARN never cleans up container directories from a full disk