Hadoop Common
  1. Hadoop Common
  2. HADOOP-1044

TestDecommission fails because it attempts to transfer block to a dead datanode

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.12.0
    • Component/s: test
    • Labels:
      None

      Description

      There are two iterations in TestDecommission. After the first iteration, one datanode will be shut down because it was decommissioned. In the second iteration, while decommissioning the node, if it attempts to transfer blocks to the shut down node, the test will fail.

      http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/29/console

      1. hadoop-1044-3.patch
        4 kB
        Wendy Chien
      2. hadoop-1044-2.patch
        4 kB
        Wendy Chien
      3. hadoop-1044.patch
        3 kB
        Wendy Chien

        Activity

        Hide
        Wendy Chien added a comment -

        This patch changes the test to keep track of the datanodes that have been decommissioned and keeps them in the exclude file. The problem with the test before was that the exclude file was overwritten with only the latest node to be decommissioned. The previously decommissioned node would then be marked as normal instead of decommissioned, making it a valid target for replication.

        Show
        Wendy Chien added a comment - This patch changes the test to keep track of the datanodes that have been decommissioned and keeps them in the exclude file. The problem with the test before was that the exclude file was overwritten with only the latest node to be decommissioned. The previously decommissioned node would then be marked as normal instead of decommissioned, making it a valid target for replication.
        Hide
        dhruba borthakur added a comment -

        +1. Code reviewed.

        One minor comment: there are 5 datanodes and in the second iteration two of the datanodes are in decommission state. Thus the test file should get replicated on the remaining three datanodes. In this is correct, you might want to make numDatanodes = numIterations + 3.

        Show
        dhruba borthakur added a comment - +1. Code reviewed. One minor comment: there are 5 datanodes and in the second iteration two of the datanodes are in decommission state. Thus the test file should get replicated on the remaining three datanodes. In this is correct, you might want to make numDatanodes = numIterations + 3.
        Hide
        Wendy Chien added a comment -

        Thanks for reviewing the patch, Dhruba! I changed numDatanodes to be numIterations+3.

        Show
        Wendy Chien added a comment - Thanks for reviewing the patch, Dhruba! I changed numDatanodes to be numIterations+3.
        Hide
        Hadoop QA added a comment -

        -1, because javac generated 772 warnings (more than the acceptable 768 warnings) when testing the latest attachment (http://issues.apache.org/jira/secure/attachment/12352145/hadoop-1044-2.patch) against trunk revision http://svn.apache.org/repos/asf/lucene/hadoop/trunk/512006. Please note that this message is automatically generated and may represent a problem with the automation system and not the patch. Results are at http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch

        Show
        Hadoop QA added a comment - -1, because javac generated 772 warnings (more than the acceptable 768 warnings) when testing the latest attachment ( http://issues.apache.org/jira/secure/attachment/12352145/hadoop-1044-2.patch ) against trunk revision http://svn.apache.org/repos/asf/lucene/hadoop/trunk/512006 . Please note that this message is automatically generated and may represent a problem with the automation system and not the patch. Results are at http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch
        Hide
        Wendy Chien added a comment -

        Removed use of generics. Trying again...

        Show
        Wendy Chien added a comment - Removed use of generics. Trying again...
        Hide
        Doug Cutting added a comment -

        I just committed this. Thanks, Wendy!

        Show
        Doug Cutting added a comment - I just committed this. Thanks, Wendy!

          People

          • Assignee:
            Wendy Chien
            Reporter:
            Wendy Chien
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development