Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9044

Give Priority to FavouredNodes , before selecting nodes from FavouredNode's Node Group

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Passing Favored nodes intention is to place replica among the favored node

      Current behavior in Node group is
      If favored node is not available it goes to one among favored nodegroup.

      Say for example:
      	1)I need 3 replicas and passed 5 favored nodes.
      	2)Out of 5 favored nodes 3 favored nodes are not good.
      	3)Then based on BlockPlacementPolicyWithNodeGroup out of 5 targets node returned , 3 will be random node from 3 bad FavoredNode's nodegroup. 
      	4)Then there is a probability that all my 3 replicas are placed on Random node from FavoredNodes's nodegroup , instead of giving priority to 2 favored nodes returned as target.
      

      Instead of returning 5 targets on 3rd step above , we can return 2 good favored nodes as target
      And remaining 1 needed replica can be chosen from Random node of bad FavoredNodes's nodegroup.
      This will make sure that the FavoredNodes are given priority.

      1. HDFS-9044.6.patch
        11 kB
        J.Andreina
      2. HDFS-9044.5.patch
        11 kB
        J.Andreina
      3. HDFS-9044.4.patch
        12 kB
        J.Andreina
      4. HDFS-9044.3.patch
        12 kB
        J.Andreina
      5. HDFS-9044.2.patch
        12 kB
        J.Andreina
      6. HDFS-9044.1.patch
        12 kB
        J.Andreina

        Activity

        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #550 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/550/)
        HDFS-9044. Give Priority to FavouredNodes , before selecting nodes from (vinayakumarb: rev 588baab160e7c328dca1c45cf3541e79218406e8)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #550 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/550/ ) HDFS-9044 . Give Priority to FavouredNodes , before selecting nodes from (vinayakumarb: rev 588baab160e7c328dca1c45cf3541e79218406e8) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2488 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2488/)
        HDFS-9044. Give Priority to FavouredNodes , before selecting nodes from (vinayakumarb: rev 588baab160e7c328dca1c45cf3541e79218406e8)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2488 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2488/ ) HDFS-9044 . Give Priority to FavouredNodes , before selecting nodes from (vinayakumarb: rev 588baab160e7c328dca1c45cf3541e79218406e8) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #612 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/612/)
        HDFS-9044. Give Priority to FavouredNodes , before selecting nodes from (vinayakumarb: rev 588baab160e7c328dca1c45cf3541e79218406e8)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #612 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/612/ ) HDFS-9044 . Give Priority to FavouredNodes , before selecting nodes from (vinayakumarb: rev 588baab160e7c328dca1c45cf3541e79218406e8) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #1335 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1335/)
        HDFS-9044. Give Priority to FavouredNodes , before selecting nodes from (vinayakumarb: rev 588baab160e7c328dca1c45cf3541e79218406e8)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1335 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1335/ ) HDFS-9044 . Give Priority to FavouredNodes , before selecting nodes from (vinayakumarb: rev 588baab160e7c328dca1c45cf3541e79218406e8) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2542 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2542/)
        HDFS-9044. Give Priority to FavouredNodes , before selecting nodes from (vinayakumarb: rev 588baab160e7c328dca1c45cf3541e79218406e8)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2542 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2542/ ) HDFS-9044 . Give Priority to FavouredNodes , before selecting nodes from (vinayakumarb: rev 588baab160e7c328dca1c45cf3541e79218406e8) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #599 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/599/)
        HDFS-9044. Give Priority to FavouredNodes , before selecting nodes from (vinayakumarb: rev 588baab160e7c328dca1c45cf3541e79218406e8)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #599 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/599/ ) HDFS-9044 . Give Priority to FavouredNodes , before selecting nodes from (vinayakumarb: rev 588baab160e7c328dca1c45cf3541e79218406e8) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8723 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8723/)
        HDFS-9044. Give Priority to FavouredNodes , before selecting nodes from (vinayakumarb: rev 588baab160e7c328dca1c45cf3541e79218406e8)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8723 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8723/ ) HDFS-9044 . Give Priority to FavouredNodes , before selecting nodes from (vinayakumarb: rev 588baab160e7c328dca1c45cf3541e79218406e8) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java
        Hide
        vinayrpet Vinayakumar B added a comment -

        Committed to trunk and branch-2.
        Thanks J.Andreina for the contribution.

        Show
        vinayrpet Vinayakumar B added a comment - Committed to trunk and branch-2. Thanks J.Andreina for the contribution.
        Hide
        vinayrpet Vinayakumar B added a comment -

        +1 for latest patch.

        Will commit shortly

        Show
        vinayrpet Vinayakumar B added a comment - +1 for latest patch. Will commit shortly
        Hide
        andreina J.Andreina added a comment -

        Failures are unrelated to this patch.
        Please review.

        Show
        andreina J.Andreina added a comment - Failures are unrelated to this patch. Please review.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 9s docker + precommit patch detected.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 3m 57s trunk passed
        +1 compile 0m 38s trunk passed with JDK v1.8.0_60
        +1 compile 0m 41s trunk passed with JDK v1.7.0_79
        +1 checkstyle 0m 18s trunk passed
        +1 mvneclipse 0m 16s trunk passed
        -1 findbugs 2m 4s hadoop-hdfs-project/hadoop-hdfs in trunk cannot run convertXmlToText from findbugs
        +1 javadoc 1m 13s trunk passed with JDK v1.8.0_60
        +1 javadoc 1m 57s trunk passed with JDK v1.7.0_79
        +1 mvninstall 0m 40s the patch passed
        +1 compile 0m 40s the patch passed with JDK v1.8.0_60
        +1 javac 0m 40s the patch passed
        +1 compile 0m 34s the patch passed with JDK v1.7.0_79
        +1 javac 0m 34s the patch passed
        -1 checkstyle 0m 18s Patch generated 2 new checkstyle issues in hadoop-hdfs-project/hadoop-hdfs (total was 51, now 52).
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 13s the patch passed
        +1 javadoc 1m 14s the patch passed with JDK v1.8.0_60
        +1 javadoc 1m 58s the patch passed with JDK v1.7.0_79
        -1 unit 56m 31s hadoop-hdfs in the patch failed with JDK v1.8.0_60.
        -1 unit 49m 27s hadoop-hdfs in the patch failed with JDK v1.7.0_79.
        -1 asflicense 0m 20s Patch generated 58 ASF License warnings.
        128m 11s



        Reason Tests
        JDK v1.7.0_79 Failed junit tests hadoop.hdfs.server.blockmanagement.TestNodeCount
          hadoop.hdfs.server.datanode.TestBlockScanner
          hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
          hadoop.hdfs.TestRecoverStripedFile



        Subsystem Report/Notes
        Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-10-29
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12769230/HDFS-9044.6.patch
        JIRA Issue HDFS-9044
        Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile
        uname Linux 322d50b7733c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-67f42f1/precommit/personality/hadoop.sh
        git revision trunk / 65f53f2
        Default Java 1.7.0_79
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
        findbugs v3.0.0
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/13266/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs.html
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13266/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/13266/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/13266/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt
        unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13266/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13266/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt
        JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13266/testReport/
        asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13266/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Max memory used 227MB
        Powered by Apache Yetus http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13266/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 9s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 3m 57s trunk passed +1 compile 0m 38s trunk passed with JDK v1.8.0_60 +1 compile 0m 41s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 18s trunk passed +1 mvneclipse 0m 16s trunk passed -1 findbugs 2m 4s hadoop-hdfs-project/hadoop-hdfs in trunk cannot run convertXmlToText from findbugs +1 javadoc 1m 13s trunk passed with JDK v1.8.0_60 +1 javadoc 1m 57s trunk passed with JDK v1.7.0_79 +1 mvninstall 0m 40s the patch passed +1 compile 0m 40s the patch passed with JDK v1.8.0_60 +1 javac 0m 40s the patch passed +1 compile 0m 34s the patch passed with JDK v1.7.0_79 +1 javac 0m 34s the patch passed -1 checkstyle 0m 18s Patch generated 2 new checkstyle issues in hadoop-hdfs-project/hadoop-hdfs (total was 51, now 52). +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 13s the patch passed +1 javadoc 1m 14s the patch passed with JDK v1.8.0_60 +1 javadoc 1m 58s the patch passed with JDK v1.7.0_79 -1 unit 56m 31s hadoop-hdfs in the patch failed with JDK v1.8.0_60. -1 unit 49m 27s hadoop-hdfs in the patch failed with JDK v1.7.0_79. -1 asflicense 0m 20s Patch generated 58 ASF License warnings. 128m 11s Reason Tests JDK v1.7.0_79 Failed junit tests hadoop.hdfs.server.blockmanagement.TestNodeCount   hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot   hadoop.hdfs.TestRecoverStripedFile Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-10-29 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12769230/HDFS-9044.6.patch JIRA Issue HDFS-9044 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile uname Linux 322d50b7733c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-67f42f1/precommit/personality/hadoop.sh git revision trunk / 65f53f2 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/13266/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13266/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13266/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13266/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13266/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13266/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13266/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13266/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 227MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13266/console This message was automatically generated.
        Hide
        vinayrpet Vinayakumar B added a comment -

        +1, pending jenkins

        Show
        vinayrpet Vinayakumar B added a comment - +1, pending jenkins
        Hide
        andreina J.Andreina added a comment -

        Thanks Vinayakumar B for the review comments.
        Updated the patch please review

        Show
        andreina J.Andreina added a comment - Thanks Vinayakumar B for the review comments. Updated the patch please review
        Hide
        vinayrpet Vinayakumar B added a comment -

        Patch looks good.
        checkstyle comments can be cleared by formatting below line

            super.chooseFavouredNodes(src, numOfReplicas, favoredNodes,
              favoriteAndExcludedNodes, blocksize,
              maxNodesPerRack, results, avoidStaleNodes, storageTypes);

        Whitespaces line endigs also could be removed.

        +1 once addresed.

        Show
        vinayrpet Vinayakumar B added a comment - Patch looks good. checkstyle comments can be cleared by formatting below line super .chooseFavouredNodes(src, numOfReplicas, favoredNodes, favoriteAndExcludedNodes, blocksize, maxNodesPerRack, results, avoidStaleNodes, storageTypes); Whitespaces line endigs also could be removed. +1 once addresed.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 18m 53s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 57s There were no new javac warning messages.
        +1 javadoc 10m 22s There were no new javadoc warning messages.
        -1 release audit 0m 20s The applied patch generated 1 release audit warnings.
        -1 checkstyle 1m 29s The applied patch generated 4 new checkstyle issues (total was 49, now 52).
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 install 1m 34s mvn install still works.
        +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse.
        +1 findbugs 2m 28s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 12s Pre-build of native portion
        -1 hdfs tests 50m 18s Tests failed in hadoop-hdfs.
            97m 11s  



        Reason Tests
        Failed unit tests hadoop.hdfs.TestDFSClientRetries
          hadoop.hdfs.TestLeaseRecovery2
          hadoop.fs.TestGlobPaths
        Timed out tests org.apache.hadoop.hdfs.tools.TestDFSAdmin



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12766714/HDFS-9044.5.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / c80b3a8
        Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12999/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
        Release Audit https://builds.apache.org/job/PreCommit-HDFS-Build/12999/artifact/patchprocess/patchReleaseAuditProblems.txt
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12999/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/12999/artifact/patchprocess/whitespace.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12999/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12999/testReport/
        Java 1.7.0_55
        uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12999/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 18m 53s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 57s There were no new javac warning messages. +1 javadoc 10m 22s There were no new javadoc warning messages. -1 release audit 0m 20s The applied patch generated 1 release audit warnings. -1 checkstyle 1m 29s The applied patch generated 4 new checkstyle issues (total was 49, now 52). -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 34s mvn install still works. +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse. +1 findbugs 2m 28s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 12s Pre-build of native portion -1 hdfs tests 50m 18s Tests failed in hadoop-hdfs.     97m 11s   Reason Tests Failed unit tests hadoop.hdfs.TestDFSClientRetries   hadoop.hdfs.TestLeaseRecovery2   hadoop.fs.TestGlobPaths Timed out tests org.apache.hadoop.hdfs.tools.TestDFSAdmin Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12766714/HDFS-9044.5.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / c80b3a8 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12999/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html Release Audit https://builds.apache.org/job/PreCommit-HDFS-Build/12999/artifact/patchprocess/patchReleaseAuditProblems.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12999/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/12999/artifact/patchprocess/whitespace.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12999/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12999/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12999/console This message was automatically generated.
        Hide
        andreina J.Andreina added a comment -

        Thanks Vinayakumar B for the review comments.
        Have updated the patch. Please review.

        Show
        andreina J.Andreina added a comment - Thanks Vinayakumar B for the review comments. Have updated the patch. Please review.
        Hide
        vinayrpet Vinayakumar B added a comment -

        Hi J.Andreina, thanks for the patch,

        Patch looks almost good.

        Here are few minor comments,

        1. Below lines from BlockPlacementPolicyWithNodeGroup#chooseFavouredNodes(..) could be just replaced with super.chooseFavouredNodes(..)

        +    for (int i = 0; i < favoredNodes.size() && results.size() < numOfReplicas;
        +        i++) {
        +      DatanodeDescriptor favoredNode = favoredNodes.get(i);
        +      // Choose a single node which is local to favoredNode.
        +      // 'results' is updated within chooseLocalNode
        +      DatanodeStorageInfo target = null;
        +      try {
        +        target =
        +            chooseLocalStorage(favoredNode, favoriteAndExcludedNodes,
        +              blocksize, maxNodesPerRack, results, avoidStaleNodes,
        +              storageTypes, false);
        +      } catch (NotEnoughReplicasException e) {
        +        // catch Exception and continue with other favored nodes
        +        continue;
        +      }
        +      if (target == null) {
        +        LOG.warn("Could not find a target for file " + src
        +            + " with favored node " + favoredNode);
        +        continue;
        +      }
        +      favoriteAndExcludedNodes.add(target.getDatanodeDescriptor());
        +    }

        2. checkFavoredNodePartOfTarget(..) could be renamed to isNodeChosen(..)

        3. clusterMapNodeGroup.getNodeGroup(favoredNode.getNetworkLocation()) can be extracted to a variable scope for better readability

        4. I think its better to update javadoc for BlockPlacementPolicyWithNodeGroup#chooseLocalStorage(..) by mentioning, fallback to nodegroup/rack will happen if flag fallbackToNodeGroupAndLocalRack is set.

        Show
        vinayrpet Vinayakumar B added a comment - Hi J.Andreina , thanks for the patch, Patch looks almost good. Here are few minor comments, 1. Below lines from BlockPlacementPolicyWithNodeGroup#chooseFavouredNodes(..) could be just replaced with super.chooseFavouredNodes(..) + for ( int i = 0; i < favoredNodes.size() && results.size() < numOfReplicas; + i++) { + DatanodeDescriptor favoredNode = favoredNodes.get(i); + // Choose a single node which is local to favoredNode. + // 'results' is updated within chooseLocalNode + DatanodeStorageInfo target = null ; + try { + target = + chooseLocalStorage(favoredNode, favoriteAndExcludedNodes, + blocksize, maxNodesPerRack, results, avoidStaleNodes, + storageTypes, false ); + } catch (NotEnoughReplicasException e) { + // catch Exception and continue with other favored nodes + continue ; + } + if (target == null ) { + LOG.warn( "Could not find a target for file " + src + + " with favored node " + favoredNode); + continue ; + } + favoriteAndExcludedNodes.add(target.getDatanodeDescriptor()); + } 2. checkFavoredNodePartOfTarget(..) could be renamed to isNodeChosen(..) 3. clusterMapNodeGroup.getNodeGroup(favoredNode.getNetworkLocation()) can be extracted to a variable scope for better readability 4. I think its better to update javadoc for BlockPlacementPolicyWithNodeGroup#chooseLocalStorage(..) by mentioning, fallback to nodegroup/rack will happen if flag fallbackToNodeGroupAndLocalRack is set.
        Hide
        andreina J.Andreina added a comment -

        Vinayakumar B and Tsz Wo Nicholas Sze , can you please have a look at this jira and provide your feedback

        Show
        andreina J.Andreina added a comment - Vinayakumar B and Tsz Wo Nicholas Sze , can you please have a look at this jira and provide your feedback
        Hide
        andreina J.Andreina added a comment -

        Checkstyle comments can be ignored , as all the 9 parameters of the method are been used.
        Please review.

        Show
        andreina J.Andreina added a comment - Checkstyle comments can be ignored , as all the 9 parameters of the method are been used. Please review.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 18m 2s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 8m 0s There were no new javac warning messages.
        +1 javadoc 10m 6s There were no new javadoc warning messages.
        +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 1m 24s The applied patch generated 2 new checkstyle issues (total was 50, now 51).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 32s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 2m 33s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 13s Pre-build of native portion
        +1 hdfs tests 164m 10s Tests passed in hadoop-hdfs.
            210m 0s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12761822/HDFS-9044.4.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 7c5c099
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12618/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12618/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12618/testReport/
        Java 1.7.0_55
        uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12618/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 2s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 8m 0s There were no new javac warning messages. +1 javadoc 10m 6s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 24s The applied patch generated 2 new checkstyle issues (total was 50, now 51). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 32s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 2m 33s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 13s Pre-build of native portion +1 hdfs tests 164m 10s Tests passed in hadoop-hdfs.     210m 0s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12761822/HDFS-9044.4.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 7c5c099 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12618/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12618/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12618/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12618/console This message was automatically generated.
        Hide
        andreina J.Andreina added a comment -

        Updated the patch.
        Please review.

        Show
        andreina J.Andreina added a comment - Updated the patch. Please review.
        Hide
        andreina J.Andreina added a comment -

        Testcase failures are unrelated.
        Checkstyle comments can be ignored , as all the 9 parameters of the method are been used.
        Please review.

        Show
        andreina J.Andreina added a comment - Testcase failures are unrelated. Checkstyle comments can be ignored , as all the 9 parameters of the method are been used. Please review.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 17m 33s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 46s There were no new javac warning messages.
        +1 javadoc 10m 1s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 1m 19s The applied patch generated 2 new checkstyle issues (total was 50, now 51).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 28s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 findbugs 2m 26s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 15s Pre-build of native portion
        -1 hdfs tests 163m 11s Tests failed in hadoop-hdfs.
            208m 0s  



        Reason Tests
        Failed unit tests hadoop.hdfs.TestDFSClientRetries
          hadoop.hdfs.server.namenode.TestFSNamesystem



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12761595/HDFS-9044.3.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 10ab7d5
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12584/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12584/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12584/testReport/
        Java 1.7.0_55
        uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12584/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 33s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 46s There were no new javac warning messages. +1 javadoc 10m 1s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 19s The applied patch generated 2 new checkstyle issues (total was 50, now 51). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 28s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 2m 26s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 15s Pre-build of native portion -1 hdfs tests 163m 11s Tests failed in hadoop-hdfs.     208m 0s   Reason Tests Failed unit tests hadoop.hdfs.TestDFSClientRetries   hadoop.hdfs.server.namenode.TestFSNamesystem Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12761595/HDFS-9044.3.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 10ab7d5 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12584/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12584/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12584/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12584/console This message was automatically generated.
        Hide
        andreina J.Andreina added a comment -

        Fixed checkstyle and findbug errors.
        Updated the patch.
        Please review.

        Show
        andreina J.Andreina added a comment - Fixed checkstyle and findbug errors. Updated the patch. Please review.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 17m 56s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 8m 8s There were no new javac warning messages.
        +1 javadoc 10m 26s There were no new javadoc warning messages.
        +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 1m 20s The applied patch generated 6 new checkstyle issues (total was 50, now 55).
        +1 whitespace 0m 1s The patch has no lines that end in whitespace.
        +1 install 1m 31s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        -1 findbugs 2m 36s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings.
        +1 native 3m 29s Pre-build of native portion
        +1 hdfs tests 189m 50s Tests passed in hadoop-hdfs.
            236m 19s  



        Reason Tests
        FindBugs module:hadoop-hdfs



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12761401/HDFS-9044.2.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / c9cb6a5
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12566/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
        Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12566/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12566/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12566/testReport/
        Java 1.7.0_55
        uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12566/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 56s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 8m 8s There were no new javac warning messages. +1 javadoc 10m 26s There were no new javadoc warning messages. +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 20s The applied patch generated 6 new checkstyle issues (total was 50, now 55). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 31s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. -1 findbugs 2m 36s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings. +1 native 3m 29s Pre-build of native portion +1 hdfs tests 189m 50s Tests passed in hadoop-hdfs.     236m 19s   Reason Tests FindBugs module:hadoop-hdfs Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12761401/HDFS-9044.2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / c9cb6a5 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12566/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12566/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12566/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12566/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12566/console This message was automatically generated.
        Hide
        andreina J.Andreina added a comment -

        Updated the patch fixing the checkstyle and javadoc warnings.
        Testcase failures are unrelated.
        Please review.

        Show
        andreina J.Andreina added a comment - Updated the patch fixing the checkstyle and javadoc warnings. Testcase failures are unrelated. Please review.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 18m 0s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 8m 1s There were no new javac warning messages.
        -1 javadoc 10m 19s The applied patch generated 1 additional warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 1m 24s The applied patch generated 10 new checkstyle issues (total was 50, now 59).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 28s mvn install still works.
        +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse.
        -1 findbugs 2m 35s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings.
        +1 native 3m 18s Pre-build of native portion
        -1 hdfs tests 193m 1s Tests failed in hadoop-hdfs.
            239m 8s  



        Reason Tests
        FindBugs module:hadoop-hdfs
        Failed unit tests hadoop.hdfs.server.namenode.TestFileTruncate
          hadoop.hdfs.TestEncryptionZonesWithKMS



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12761367/HDFS-9044.1.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / c9cb6a5
        javadoc https://builds.apache.org/job/PreCommit-HDFS-Build/12564/artifact/patchprocess/diffJavadocWarnings.txt
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12564/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
        Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12564/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12564/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12564/testReport/
        Java 1.7.0_55
        uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12564/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 0s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 8m 1s There were no new javac warning messages. -1 javadoc 10m 19s The applied patch generated 1 additional warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 24s The applied patch generated 10 new checkstyle issues (total was 50, now 59). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 28s mvn install still works. +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse. -1 findbugs 2m 35s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings. +1 native 3m 18s Pre-build of native portion -1 hdfs tests 193m 1s Tests failed in hadoop-hdfs.     239m 8s   Reason Tests FindBugs module:hadoop-hdfs Failed unit tests hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.TestEncryptionZonesWithKMS Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12761367/HDFS-9044.1.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / c9cb6a5 javadoc https://builds.apache.org/job/PreCommit-HDFS-Build/12564/artifact/patchprocess/diffJavadocWarnings.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12564/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12564/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12564/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12564/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12564/console This message was automatically generated.
        Hide
        andreina J.Andreina added a comment -

        Updated an initial patch ( for choosing good favored nodes as target with higher priority , before choosing bad FavoredNode's node group)

        Please review the patch and give your feedback.

        Show
        andreina J.Andreina added a comment - Updated an initial patch ( for choosing good favored nodes as target with higher priority , before choosing bad FavoredNode's node group) Please review the patch and give your feedback.

          People

          • Assignee:
            andreina J.Andreina
            Reporter:
            andreina J.Andreina
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development