Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-637

DataNode sends a Success ack when block write fails

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: datanode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When I work on HDFS-624, I saw TestFileAppend3#TC7 occasionally fails. After lots of debug, I saw that the client unexpected received a response of "-2 SUCCESS SUCCESS" in which -2 is the packet sequence number. This happened in a pipeline of 2 datanodes and one of them failed. It turned out when block receiver fails, it shuts down itself and interrupts the packet responder but responder tries to handle interruption with the condition "Thread.isInterrupted()" but unfortunately a thread's interrupt status is not set in some cases as explained in the Thread#interrupt javadoc:

      If this thread is blocked in an invocation of the wait(), wait(long), or wait(long, int) methods of the Object class, or of the join(), join(long), join(long, int), sleep(long), or sleep(long, int), methods of this class, then its interrupt status will be cleared and it will receive an InterruptedException.

      So datanode does not detect the interruption and continues as if no error occurs.

      1. interrupted.patch
        1 kB
        Hairong Kuang
      2. interrupted1.patch
        2 kB
        Hairong Kuang

        Activity

        Tom White made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hairong Kuang made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hairong Kuang made changes -
        Summary DataNode sends an Success ack when block write fails DataNode sends a Success ack when block write fails
        Hadoop Flags [Reviewed]
        Hairong Kuang made changes -
        Attachment interrupted1.patch [ 12420324 ]
        Hairong Kuang made changes -
        Field Original Value New Value
        Attachment interrupted.patch [ 12420312 ]
        Hairong Kuang created issue -

          People

          • Assignee:
            Hairong Kuang
            Reporter:
            Hairong Kuang
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development