[HDFS-15574] Remove unnecessary sort of block list in DirectoryScanner

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.2.2, 3.3.1, 3.4.0, 3.2.3
• Component/s: datanode
• Target Version/s: 3.4.0
• Hadoop Flags: Reviewed
• Assignee: Stephen O'Donnell
• Reporter: Stephen O'Donnell
• Created: 11/Sep/20 09:46
• Updated: 27/Jan/24 03:09
• Resolved: 17/Sep/20 04:51

Description

These lines of code in DirectoryScanner#scan(), obtain a snapshot of the finalized blocks from memory, and then sort them, under the DN lock. However the blocks are stored in a sorted structure (FoldedTreeSet) and hence the sort should be unnecessary.

  final List<ReplicaInfo> bl = dataset.getFinalizedBlocks(bpid);
  Collections.sort(bl); // Sort based on blockId

This Jira removes the sort, and renames the getFinalizedBlocks to getSortedFinalizedBlocks to make the intent of the method more clear.

Also added a test, just in case the underlying block structure is ever changed to something unsorted.

Issue Links

• relates to: HDFS-13671 Namenode deletes large dir slowly caused by FoldedTreeSet#removeAndGet