Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-6602

PendingDeletionBlocks on SBN keeps increasing

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Critical Critical
    • Resolution: Duplicate
    • Affects Version/s: 2.5.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Target Version/s:

      Description

      PendingDeletionBlocks is from BlockManager.invalidateBlocks.numBlocks(). It
      means this data structure is populated, but IBR (incremental block reports) do not cause deleted blocks to be removed from it. As a result, the heap usage keeps increasing.

        Issue Links

          Activity

          Hide
          Kihwal Lee added a comment -

          It's already fixed by HDFS-6424!

          Show
          Kihwal Lee added a comment - It's already fixed by HDFS-6424 !
          Hide
          Kihwal Lee added a comment -

          Block report processing is actually okay. All types of block report processing goes through BlockManager#processReportedBlock() and any report from future is queued. It is delete() that causes this queue to be populated. After collecting all blocks to be invalidated, BlockManager#removeBlock() is called, which calls addToInvalidates(). If NN is in standby, addToInvalidates() should not be called.

          Show
          Kihwal Lee added a comment - Block report processing is actually okay. All types of block report processing goes through BlockManager#processReportedBlock() and any report from future is queued. It is delete() that causes this queue to be populated. After collecting all blocks to be invalidated, BlockManager#removeBlock() is called, which calls addToInvalidates() . If NN is in standby, addToInvalidates() should not be called.
          Hide
          Kihwal Lee added a comment - - edited

          Since ReplicationMonitor is not running or not generating any work on SBN, invalidateBlocks is not consumed. Only when SBN becomes active, it will be cleared. invalidateBlocks is populated during block report processing. I think no queues including invalidateBlocks should be populated in standby.

          Show
          Kihwal Lee added a comment - - edited Since ReplicationMonitor is not running or not generating any work on SBN, invalidateBlocks is not consumed. Only when SBN becomes active, it will be cleared. invalidateBlocks is populated during block report processing. I think no queues including invalidateBlocks should be populated in standby.

            People

            • Assignee:
              Kihwal Lee
              Reporter:
              Kihwal Lee
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development