Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-611

Heartbeats times from Datanodes increase when there are plenty of blocks to delete

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.20.1, 0.21.0, 0.22.0
    • 0.21.0, 1.0.0
    • datanode
    • None
    • Reviewed

    Description

      I am seeing that when we delete a large directory that has plenty of blocks, the heartbeat times from datanodes increase significantly from the normal value of 3 seconds to as large as 50 seconds or so. The heartbeat thread in the Datanode deletes a bunch of blocks sequentially, this causes the heartbeat times to increase.

      Attachments

        1. HDFS-611.branch-19.patch
          8 kB
          Zheng Shao
        2. HDFS-611.branch-19.v2.patch
          8 kB
          Zheng Shao
        3. HDFS-611.branch-20.patch
          8 kB
          Zheng Shao
        4. HDFS-611.branch-20.v2.patch
          8 kB
          Zheng Shao
        5. HDFS-611.branch-20.v6.patch
          14 kB
          Zheng Shao
        6. HDFS-611.trunk.patch
          8 kB
          Zheng Shao
        7. HDFS-611.trunk.v2.patch
          8 kB
          Zheng Shao
        8. HDFS-611.trunk.v3.patch
          12 kB
          Zheng Shao
        9. HDFS-611.trunk.v4.patch
          15 kB
          Zheng Shao
        10. HDFS-611.trunk.v5.patch
          15 kB
          Zheng Shao
        11. HDFS-611.trunk.v6.patch
          15 kB
          Zheng Shao
        12. HDFS-611-branch-0.20-security.patch
          14 kB
          Jitendra Nath Pandey

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            zshao Zheng Shao Assign to me
            dhruba Dhruba Borthakur
            Votes:
            0 Vote for this issue
            Watchers:
            14 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment