[HDFS-15354] clearCorruptLazyPersistFiles incrementalBlock removal should be out side write lock

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.2.1
• Component/s: namenode
• Target Version/s: 3.5.0
• Assignee: Uma Maheswara Rao G
• Reporter: Uma Maheswara Rao G
• Created: 13/May/20 19:15
• Updated: 04/Jan/24 08:13

Description

 In LazyPersistFileScrubber#clearCorruptLazyPersistFiles collecting blocks for removal and also removing them in write lock.

removeBlocks should be moved out of writelock as removeBlocks has incremental deletion logic in which it will acquire write lock and unlock for every block removal.

If there are more corrupt blocks to remove in cluster, it may hold write lock for longer time. 

  for (BlockCollection bc : filesToDelete) {
          LOG.warn("Removing lazyPersist file " + bc.getName() + " with no replicas.");
          BlocksMapUpdateInfo toRemoveBlocks =
              FSDirDeleteOp.deleteInternal(
                  FSNamesystem.this,
                  INodesInPath.fromINode((INodeFile) bc), false);
          changed |= toRemoveBlocks != null;
          if (toRemoveBlocks != null) {
            removeBlocks(toRemoveBlocks); // Incremental deletion of blocks
          }
        }