Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-4679

Datanode prints tons of log messages: Waiting for threadgroup to exit, active theads is XX

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.18.3
    • None
    • None
    • Reviewed
    • Hide
      1. Only datanode's offerService thread shutdown the datanode to avoid deadlock;
      2. Datanode checks disk in case of failure on creating a block file.
      Show
      1. Only datanode's offerService thread shutdown the datanode to avoid deadlock; 2. Datanode checks disk in case of failure on creating a block file.

    Description

      When a data receiver thread sees a disk error, it immediately calls shutdown to shutdown DataNode. But the shutdown method does not return before all data receiver threads exit, which will never happen. Therefore the DataNode gets into a dead/live lock state, emitting tons of log messages: Waiting for threadgroup to exit, active threads is XX.

      Attachments

        1. diskError.patch
          6 kB
          Hairong Kuang
        2. diskError1.patch
          6 kB
          Hairong Kuang
        3. diskError2.patch
          8 kB
          Hairong Kuang
        4. diskError3.patch
          9 kB
          Hairong Kuang
        5. diskError3-br18.patch
          7 kB
          Hairong Kuang

        Issue Links

          Activity

            People

              hairong Hairong Kuang
              hairong Hairong Kuang
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: