Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9910

Datanode heartbeats can get blocked by disk in FsDatasetImpl#checkBlock()

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 2.7.2
    • None
    • datanode
    • None

    Description

      When a data node needs to transfer a block, it validates the block in the heartbeat thread invoking the checkBlock() method of FsDatasetImpl, where it checks whether the block exists and gets the block length. If the block is valid, it then spins off a thread to do the actual block transfer. We found that during heavy disk IO the heartbeat thread hangs on replicaInfo.getBlockFile().exists() for more than 10 minutes.

      Attachments

        Issue Links

          duplicates

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-9901 Move disk IO out of the heartbeat thread

          • Major - Major loss of function.
          • Patch Available
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-9882 Add heartbeatsTotal in Datanode metrics

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            People

              hualiu Hua Liu
              elgoiri Íñigo Goiri
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: