[HADOOP-15450] Avoid fsync storm triggered by DiskChecker and handle disk full situation

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 3.1.0, 2.10.0, 3.2.0, 3.1.1, 2.9.2, 3.0.3, 2.8.5
• Target Version/s: 3.1.1, 2.9.2, 3.0.3, 2.8.5
• Hadoop Flags: Reviewed
• Assignee: Arpit Agarwal
• Reporter: Kihwal Lee
• Created: 07/May/18 16:50
• Updated: 07/Nov/18 01:34
• Resolved: 22/May/18 16:38

Description

Fix disk checker issues reported by kihwal in HADOOP-13738

There are non-hdfs users of DiskChecker, who use it proactively, not just on failures. This was fine before, but now it incurs heavy I/O due to introduction of fsync() in the code.

Issue Links

• blocks: HDFS-13538 HDFS DiskChecker should handle disk full situation
• is broken by: HADOOP-13738 DiskChecker should perform some disk IO