Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-4584

Slow generation of blockReport at DataNode causes delay of sending heartbeat to NameNode

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      sometimes due to disk or some other problems, datanode takes minutes or tens of minutes to generate a block report. It causes the datanode not able to send heartbeat to NameNode every 3 seconds. In the worst case, it makes NameNode to detect a lost heartbeat and wrongly decide that the datanode is dead.

      It would be nice to have two threads instead. One thread is for scanning data directories and generating block report, and executes the requests sent by NameNode; Another thread is for sending heartbeats, block reports, and picking up the requests from NameNode. By having these two threads, the sending of heartbeats will not get delayed by any slow block report or slow execution of NameNode requests.

        Attachments

        1. 4584.brthread.5.patch
          50 kB
          Suresh Srinivas
        2. 4584.brthread.5.patch
          50 kB
          Suresh Srinivas
        3. 4584.brthread.4.patch
          49 kB
          Suresh Srinivas
        4. 4584.brthread.4.patch
          49 kB
          Suresh Srinivas
        5. 4584.brthread.4.patch
          50 kB
          Suresh Srinivas
        6. Design.pdf
          69 kB
          Suresh Srinivas
        7. Design.pdf
          65 kB
          Suresh Srinivas
        8. 4584.brthread.3.patch
          47 kB
          Suresh Srinivas
        9. 4584.brthread.3.patch
          46 kB
          Suresh Srinivas
        10. 4584.brthread.3.patch
          55 kB
          Suresh Srinivas
        11. 4584.brthread.3.patch
          54 kB
          Suresh Srinivas
        12. 4584.brthread.3.patch
          29 kB
          Suresh Srinivas
        13. 4584.brthread.2.patch
          54 kB
          Suresh Srinivas
        14. 4584.hbthread.patch
          20 kB
          Suresh Srinivas
        15. 4584.patch
          29 kB
          Suresh Srinivas
        16. 4584.patch
          18 kB
          Suresh Srinivas
        17. 4584.patch
          18 kB
          Suresh Srinivas
        18. 4584.patch
          18 kB
          Suresh Srinivas
        19. 4584.patch
          15 kB
          Suresh Srinivas
        20. 4584.patch
          16 kB
          Suresh Srinivas

          Issue Links

            Activity

              People

              • Assignee:
                sureshms Suresh Srinivas
                Reporter:
                hairong Hairong Kuang
              • Votes:
                0 Vote for this issue
                Watchers:
                15 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: