Hadoop Common
  1. Hadoop Common
  2. HADOOP-4703

DataNode.createInterDataNodeProtocolProxy should not wait for proxy forever while recovering lease

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.18.0
    • Fix Version/s: 0.18.3
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The problem is that the list of DataNodes may contain DataNode that has already shutdown as the case in TestFileCreationClient.testClientTriggeredLeaseRecovery(). Using waitForProxy causes the DataNode to wait forever for the dead DataNode.

      1. 4703_20081120_0.18.patch
        1 kB
        Tsz Wo Nicholas Sze
      2. 4073_20081120.patch
        1 kB
        Tsz Wo Nicholas Sze

        Issue Links

          Activity

          Hairong Kuang created issue -
          Hairong Kuang made changes -
          Field Original Value New Value
          Link This issue blocks HADOOP-4659 [ HADOOP-4659 ]
          Hide
          Tsz Wo Nicholas Sze added a comment -

          4073_20081120.patch: change waitForProxy(...) to getProxy(...) in creating InterDatanodeProtocol and ClientDatanodeProtocol.

          waitForProxy(...) waits forever if there are ConnectException. It seems not right since if the server in the other side is down, the client cannot detect it but keep waiting.

          Show
          Tsz Wo Nicholas Sze added a comment - 4073_20081120.patch: change waitForProxy(...) to getProxy(...) in creating InterDatanodeProtocol and ClientDatanodeProtocol. waitForProxy(...) waits forever if there are ConnectException. It seems not right since if the server in the other side is down, the client cannot detect it but keep waiting.
          Tsz Wo Nicholas Sze made changes -
          Attachment 4073_20081120.patch [ 12394386 ]
          Hide
          Hairong Kuang added a comment -

          +1 This fixed the hanging TestFileCreationClient problem that we saw while testing the patch to HADOOP-4659.

          Show
          Hairong Kuang added a comment - +1 This fixed the hanging TestFileCreationClient problem that we saw while testing the patch to HADOOP-4659 .
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Since the change is simple, no new tests added.

          Show
          Tsz Wo Nicholas Sze added a comment - Since the change is simple, no new tests added.
          Tsz Wo Nicholas Sze made changes -
          Assignee Tsz Wo (Nicholas), SZE [ szetszwo ]
          Hadoop Flags [Reviewed]
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Tsz Wo Nicholas Sze added a comment -
               [exec] -1 overall.  
          
               [exec]     +1 @author.  The patch does not contain any @author tags.
          
               [exec]     -1 tests included.  The patch doesn't appear to include any new or modified tests.
               [exec]                         Please justify why no tests are needed for this patch.
          
               [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
          
               [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
          
               [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
          
               [exec]     +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
          

          Running unit tests locally.

          Show
          Tsz Wo Nicholas Sze added a comment - [exec] -1 overall. [exec] +1 @author. The patch does not contain any @author tags. [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no tests are needed for this patch. [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity. Running unit tests locally.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Passed all unit tests locally. I am going to commit this.

          Show
          Tsz Wo Nicholas Sze added a comment - Passed all unit tests locally. I am going to commit this.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          4703_20081120_0.18.patch: for 0.18

          I hate that I typed an incorrect jira number in my previous file name.

          Show
          Tsz Wo Nicholas Sze added a comment - 4703_20081120_0.18.patch: for 0.18 I hate that I typed an incorrect jira number in my previous file name.
          Tsz Wo Nicholas Sze made changes -
          Attachment 4703_20081120_0.18.patch [ 12394465 ]
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I just committed this.

          Show
          Tsz Wo Nicholas Sze added a comment - I just committed this.
          Tsz Wo Nicholas Sze made changes -
          Resolution Fixed [ 1 ]
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-trunk #668 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/668/)
          . Should not wait for proxy forever in lease recovering. (szetszwo)

          Show
          Hudson added a comment - Integrated in Hadoop-trunk #668 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/668/ ) . Should not wait for proxy forever in lease recovering. (szetszwo)
          Nigel Daley made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Owen O'Malley made changes -
          Component/s dfs [ 12310710 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          17h 35m 1 Tsz Wo Nicholas Sze 21/Nov/08 18:50
          Patch Available Patch Available Resolved Resolved
          4h 56m 1 Tsz Wo Nicholas Sze 21/Nov/08 23:46
          Resolved Resolved Closed Closed
          69d 20h 28m 1 Nigel Daley 30/Jan/09 20:14

            People

            • Assignee:
              Tsz Wo Nicholas Sze
              Reporter:
              Hairong Kuang
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development