[HDFS-15583] Backport DirectoryScanner improvements HDFS-14476, HDFS-14751 and HDFS-15048 to branch 3.2 and 3.1

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.2.0, 3.2.1
• Fix Version/s: 3.2.2, 3.2.3
• Component/s: datanode
• Target Version/s: 3.2.3
• Assignee: Stephen O'Donnell
• Reporter: Stephen O'Donnell
• Created: 17/Sep/20 15:40
• Updated: 10/Jun/21 07:43
• Resolved: 22/Sep/20 10:14

Description

HDFS-14476, HDFS-14751 and HDFS-15048 made some good improvements to the datanode DirectoryScanner, but due to a large refactor on that class in branch-3.3, they are not trivial to backport to earlier branches.

HDFS-14476 introduced the problem in HDFS-14751 and a findbugs warning, fixed in HDFS-15048, so these 3 need to be backported together.