Details

    • Hadoop Flags:
      Reviewed

      Description

      These are policies at the Router that do not require maintaing state across choices (e.g., weighted random).

      1. YARN-5324.01.patch
        28 kB
        Carlo Curino
      2. YARN-5324.02.patch
        24 kB
        Carlo Curino
      3. YARN-5324.03.patch
        48 kB
        Carlo Curino
      4. YARN-5324.04.patch
        48 kB
        Carlo Curino
      5. YARN-5324.05.patch
        48 kB
        Carlo Curino
      6. YARN-5324-YARN-2915.06.patch
        48 kB
        Carlo Curino
      7. YARN-5324-YARN-2915.07.patch
        84 kB
        Carlo Curino
      8. YARN-5324-YARN-2915.08.patch
        88 kB
        Carlo Curino
      9. YARN-5324-YARN-2915.09.patch
        88 kB
        Carlo Curino
      10. YARN-5324-YARN-2915.10.patch
        79 kB
        Carlo Curino
      11. YARN-5324-YARN-2915.11.patch
        78 kB
        Carlo Curino
      12. YARN-5324-YARN-2915.12.patch
        80 kB
        Carlo Curino
      13. YARN-5324-YARN-2915.13.patch
        80 kB
        Carlo Curino
      14. YARN-5324-YARN-2915.14.patch
        85 kB
        Carlo Curino
      15. YARN-5324-YARN-2915.15.patch
        84 kB
        Carlo Curino
      16. YARN-5324-YARN-2915.16.patch
        83 kB
        Carlo Curino

        Issue Links

          Activity

          Hide
          curino Carlo Curino added a comment -

          Thanks Subru Krishnan for reviewing and committing this.

          Show
          curino Carlo Curino added a comment - Thanks Subru Krishnan for reviewing and committing this.
          Hide
          subru Subru Krishnan added a comment -

          Thanks Carlo Curino for the patch. I fixed the minor checkstyle issues and committed to branch YARN-2915.

          Show
          subru Subru Krishnan added a comment - Thanks Carlo Curino for the patch. I fixed the minor checkstyle issues and committed to branch YARN-2915 .
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.
          +1 mvninstall 9m 40s YARN-2915 passed
          +1 compile 0m 20s YARN-2915 passed
          +1 checkstyle 0m 13s YARN-2915 passed
          +1 mvnsite 0m 25s YARN-2915 passed
          +1 mvneclipse 0m 13s YARN-2915 passed
          +1 findbugs 0m 44s YARN-2915 passed
          +1 javadoc 0m 18s YARN-2915 passed
          +1 mvninstall 0m 19s the patch passed
          +1 compile 0m 17s the patch passed
          +1 javac 0m 17s the patch passed
          -1 checkstyle 0m 10s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
          +1 mvnsite 0m 20s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 46s the patch passed
          +1 javadoc 0m 14s the patch passed
          +1 unit 0m 42s hadoop-yarn-server-common in the patch passed.
          +1 asflicense 0m 16s The patch does not generate ASF License warnings.
          16m 1s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829947/YARN-5324-YARN-2915.16.patch
          JIRA Issue YARN-5324
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 3c81c4d4dad5 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision YARN-2915 / 9abc7da
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13193/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13193/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13193/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 7 new or modified test files. +1 mvninstall 9m 40s YARN-2915 passed +1 compile 0m 20s YARN-2915 passed +1 checkstyle 0m 13s YARN-2915 passed +1 mvnsite 0m 25s YARN-2915 passed +1 mvneclipse 0m 13s YARN-2915 passed +1 findbugs 0m 44s YARN-2915 passed +1 javadoc 0m 18s YARN-2915 passed +1 mvninstall 0m 19s the patch passed +1 compile 0m 17s the patch passed +1 javac 0m 17s the patch passed -1 checkstyle 0m 10s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) +1 mvnsite 0m 20s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 46s the patch passed +1 javadoc 0m 14s the patch passed +1 unit 0m 42s hadoop-yarn-server-common in the patch passed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 16m 1s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829947/YARN-5324-YARN-2915.16.patch JIRA Issue YARN-5324 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 3c81c4d4dad5 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision YARN-2915 / 9abc7da Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13193/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13193/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common Console output https://builds.apache.org/job/PreCommit-YARN-Build/13193/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          curino Carlo Curino added a comment -

          Subru Krishnan thanks for reviewing. I addressed the nits.

          Show
          curino Carlo Curino added a comment - Subru Krishnan thanks for reviewing. I addressed the nits.
          Hide
          subru Subru Krishnan added a comment -

          +1 on the latest patch, thanks Carlo Curino for addressing all my comments.

          I have a couple of minor nits:

          • We should slf4j for logging in WeightedPolicyInfo.
          • bb.duplicate is redundant in WeightedPolicyInfo.
          • I think we can move the testNoSubClusters to the BaseFederationRouterPoliciesTest.
          Show
          subru Subru Krishnan added a comment - +1 on the latest patch, thanks Carlo Curino for addressing all my comments. I have a couple of minor nits: We should slf4j for logging in WeightedPolicyInfo . bb.duplicate is redundant in WeightedPolicyInfo . I think we can move the testNoSubClusters to the BaseFederationRouterPoliciesTest .
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.
          +1 mvninstall 10m 46s YARN-2915 passed
          +1 compile 0m 23s YARN-2915 passed
          +1 checkstyle 0m 14s YARN-2915 passed
          +1 mvnsite 0m 28s YARN-2915 passed
          +1 mvneclipse 0m 15s YARN-2915 passed
          +1 findbugs 0m 49s YARN-2915 passed
          +1 javadoc 0m 19s YARN-2915 passed
          +1 mvninstall 0m 23s the patch passed
          +1 compile 0m 19s the patch passed
          +1 javac 0m 19s the patch passed
          +1 checkstyle 0m 11s the patch passed
          +1 mvnsite 0m 24s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 56s the patch passed
          +1 javadoc 0m 14s the patch passed
          +1 unit 0m 47s hadoop-yarn-server-common in the patch passed.
          +1 asflicense 0m 16s The patch does not generate ASF License warnings.
          17m 55s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829712/YARN-5324-YARN-2915.15.patch
          JIRA Issue YARN-5324
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 5eb60682d293 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision YARN-2915 / 9abc7da
          Default Java 1.8.0_101
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13183/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13183/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 7 new or modified test files. +1 mvninstall 10m 46s YARN-2915 passed +1 compile 0m 23s YARN-2915 passed +1 checkstyle 0m 14s YARN-2915 passed +1 mvnsite 0m 28s YARN-2915 passed +1 mvneclipse 0m 15s YARN-2915 passed +1 findbugs 0m 49s YARN-2915 passed +1 javadoc 0m 19s YARN-2915 passed +1 mvninstall 0m 23s the patch passed +1 compile 0m 19s the patch passed +1 javac 0m 19s the patch passed +1 checkstyle 0m 11s the patch passed +1 mvnsite 0m 24s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 56s the patch passed +1 javadoc 0m 14s the patch passed +1 unit 0m 47s hadoop-yarn-server-common in the patch passed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 17m 55s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829712/YARN-5324-YARN-2915.15.patch JIRA Issue YARN-5324 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 5eb60682d293 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision YARN-2915 / 9abc7da Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13183/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common Console output https://builds.apache.org/job/PreCommit-YARN-Build/13183/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          curino Carlo Curino added a comment -

          fixed checkstyles

          Show
          curino Carlo Curino added a comment - fixed checkstyles
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 21s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.
          +1 mvninstall 9m 16s YARN-2915 passed
          +1 compile 0m 24s YARN-2915 passed
          +1 checkstyle 0m 17s YARN-2915 passed
          +1 mvnsite 0m 27s YARN-2915 passed
          +1 mvneclipse 0m 16s YARN-2915 passed
          +1 findbugs 0m 54s YARN-2915 passed
          +1 javadoc 0m 21s YARN-2915 passed
          +1 mvninstall 0m 26s the patch passed
          +1 compile 0m 24s the patch passed
          +1 javac 0m 24s the patch passed
          -1 checkstyle 0m 13s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common: The patch generated 6 new + 0 unchanged - 0 fixed = 6 total (was 0)
          +1 mvnsite 0m 28s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 7s the patch passed
          +1 javadoc 0m 15s the patch passed
          +1 unit 0m 52s hadoop-yarn-server-common in the patch passed.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          17m 18s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829706/YARN-5324-YARN-2915.14.patch
          JIRA Issue YARN-5324
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux e21bfcf22a96 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision YARN-2915 / 9abc7da
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13182/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13182/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13182/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 7 new or modified test files. +1 mvninstall 9m 16s YARN-2915 passed +1 compile 0m 24s YARN-2915 passed +1 checkstyle 0m 17s YARN-2915 passed +1 mvnsite 0m 27s YARN-2915 passed +1 mvneclipse 0m 16s YARN-2915 passed +1 findbugs 0m 54s YARN-2915 passed +1 javadoc 0m 21s YARN-2915 passed +1 mvninstall 0m 26s the patch passed +1 compile 0m 24s the patch passed +1 javac 0m 24s the patch passed -1 checkstyle 0m 13s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common: The patch generated 6 new + 0 unchanged - 0 fixed = 6 total (was 0) +1 mvnsite 0m 28s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 7s the patch passed +1 javadoc 0m 15s the patch passed +1 unit 0m 52s hadoop-yarn-server-common in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 17m 18s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829706/YARN-5324-YARN-2915.14.patch JIRA Issue YARN-5324 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e21bfcf22a96 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision YARN-2915 / 9abc7da Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13182/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13182/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common Console output https://builds.apache.org/job/PreCommit-YARN-Build/13182/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          curino Carlo Curino added a comment -

          Subru Krishnan thanks for the follow up.

          I think I have address all your asks with the exception of the suggestion around adding selectSubcluster.
          After implementing the other refactoring you suggested the getHomeSubcluster() implementations are
          down to 5-30 lines and an external method call I don't think will improve readabiliy by much.

          Show
          curino Carlo Curino added a comment - Subru Krishnan thanks for the follow up. I think I have address all your asks with the exception of the suggestion around adding selectSubcluster . After implementing the other refactoring you suggested the getHomeSubcluster() implementations are down to 5-30 lines and an external method call I don't think will improve readabiliy by much.
          Hide
          subru Subru Krishnan added a comment - - edited

          Thanks Carlo Curino for addressing my comments.

          The patch looks very close, have a few follow up comments:

          • PriorityRouterPolicy seems to be missing in the latest version.
          • In BaseWeightedRouterPolicy, use a Logger instead of e.printStackTrace.
          • Are we handling the null case for policyInfo in BaseWeightedRouterPolicy?
          • check for active subclusters is indeed somewhat repeated

            In that case, we should have a base version in BaseWeightedRouterPolicy which others can override in case they have acustom logic.

          • The suggestion of adding selectSubCluster is not for API purposes but purely for readability as every RouterPolicy has the same pattern.
          • Rename BaseFederationPoliciesTest to BaseFederationRouterPoliciesTest
          • Why can't we move testNoSubclusters to BaseFederationRouterPoliciesTest?
          • In all/most tests the set of "activeSubclusters" is chosen to be a subset of the one specified in the policy weights. All policies are basically stateless, previous decisions should not affect following ones so the multi invocation tests are only relevant if we check statistical properties

            IIUC then, the Javadocs Generate large number of randomized tests in tests seem misleading, can you update.

          • Some of the method in FederationPoliciesTestUtil are used by the upcoming patches for AMRMProxy (I was trying to avoid editing that class over and over at every patch).

            We should only have related changes in the patch. Editing same files incrementally over multiple patches is the norm as otherwise we will loose track of provenance which is required for selective cherry-picking, roll-backs etc.

          Show
          subru Subru Krishnan added a comment - - edited Thanks Carlo Curino for addressing my comments. The patch looks very close, have a few follow up comments: PriorityRouterPolicy seems to be missing in the latest version. In BaseWeightedRouterPolicy , use a Logger instead of e.printStackTrace . Are we handling the null case for policyInfo in BaseWeightedRouterPolicy ? check for active subclusters is indeed somewhat repeated In that case, we should have a base version in BaseWeightedRouterPolicy which others can override in case they have acustom logic. The suggestion of adding selectSubCluster is not for API purposes but purely for readability as every RouterPolicy has the same pattern. Rename BaseFederationPoliciesTest to BaseFederationRouterPoliciesTest Why can't we move testNoSubclusters to BaseFederationRouterPoliciesTest ? In all/most tests the set of "activeSubclusters" is chosen to be a subset of the one specified in the policy weights. All policies are basically stateless, previous decisions should not affect following ones so the multi invocation tests are only relevant if we check statistical properties IIUC then, the Javadocs Generate large number of randomized tests in tests seem misleading, can you update. Some of the method in FederationPoliciesTestUtil are used by the upcoming patches for AMRMProxy (I was trying to avoid editing that class over and over at every patch). We should only have related changes in the patch. Editing same files incrementally over multiple patches is the norm as otherwise we will loose track of provenance which is required for selective cherry-picking, roll-backs etc.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 25s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
          +1 mvninstall 7m 10s YARN-2915 passed
          +1 compile 0m 22s YARN-2915 passed
          +1 checkstyle 0m 13s YARN-2915 passed
          +1 mvnsite 0m 25s YARN-2915 passed
          +1 mvneclipse 0m 13s YARN-2915 passed
          +1 findbugs 0m 46s YARN-2915 passed
          +1 javadoc 0m 17s YARN-2915 passed
          +1 mvninstall 0m 20s the patch passed
          +1 compile 0m 20s the patch passed
          +1 javac 0m 20s the patch passed
          +1 checkstyle 0m 10s the patch passed
          +1 mvnsite 0m 22s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 51s the patch passed
          +1 javadoc 0m 14s the patch passed
          +1 unit 0m 43s hadoop-yarn-server-common in the patch passed.
          +1 asflicense 0m 15s The patch does not generate ASF License warnings.
          13m 57s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829261/YARN-5324-YARN-2915.13.patch
          JIRA Issue YARN-5324
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 49c709d381a9 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision YARN-2915 / 9abc7da
          Default Java 1.8.0_101
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13153/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13153/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 25s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files. +1 mvninstall 7m 10s YARN-2915 passed +1 compile 0m 22s YARN-2915 passed +1 checkstyle 0m 13s YARN-2915 passed +1 mvnsite 0m 25s YARN-2915 passed +1 mvneclipse 0m 13s YARN-2915 passed +1 findbugs 0m 46s YARN-2915 passed +1 javadoc 0m 17s YARN-2915 passed +1 mvninstall 0m 20s the patch passed +1 compile 0m 20s the patch passed +1 javac 0m 20s the patch passed +1 checkstyle 0m 10s the patch passed +1 mvnsite 0m 22s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 51s the patch passed +1 javadoc 0m 14s the patch passed +1 unit 0m 43s hadoop-yarn-server-common in the patch passed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 13m 57s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829261/YARN-5324-YARN-2915.13.patch JIRA Issue YARN-5324 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 49c709d381a9 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision YARN-2915 / 9abc7da Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13153/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common Console output https://builds.apache.org/job/PreCommit-YARN-Build/13153/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
          +1 mvninstall 7m 47s YARN-2915 passed
          +1 compile 0m 23s YARN-2915 passed
          +1 checkstyle 0m 12s YARN-2915 passed
          +1 mvnsite 0m 24s YARN-2915 passed
          +1 mvneclipse 0m 14s YARN-2915 passed
          +1 findbugs 0m 46s YARN-2915 passed
          +1 javadoc 0m 17s YARN-2915 passed
          +1 mvninstall 0m 21s the patch passed
          +1 compile 0m 20s the patch passed
          +1 javac 0m 20s the patch passed
          -1 checkstyle 0m 11s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
          +1 mvnsite 0m 23s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 55s the patch passed
          +1 javadoc 0m 14s the patch passed
          +1 unit 0m 43s hadoop-yarn-server-common in the patch passed.
          +1 asflicense 0m 16s The patch does not generate ASF License warnings.
          14m 33s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829261/YARN-5324-YARN-2915.13.patch
          JIRA Issue YARN-5324
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 2587219cb96f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision YARN-2915 / b8d9062
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13152/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13152/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13152/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files. +1 mvninstall 7m 47s YARN-2915 passed +1 compile 0m 23s YARN-2915 passed +1 checkstyle 0m 12s YARN-2915 passed +1 mvnsite 0m 24s YARN-2915 passed +1 mvneclipse 0m 14s YARN-2915 passed +1 findbugs 0m 46s YARN-2915 passed +1 javadoc 0m 17s YARN-2915 passed +1 mvninstall 0m 21s the patch passed +1 compile 0m 20s the patch passed +1 javac 0m 20s the patch passed -1 checkstyle 0m 11s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) +1 mvnsite 0m 23s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 55s the patch passed +1 javadoc 0m 14s the patch passed +1 unit 0m 43s hadoop-yarn-server-common in the patch passed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 14m 33s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829261/YARN-5324-YARN-2915.13.patch JIRA Issue YARN-5324 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 2587219cb96f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision YARN-2915 / b8d9062 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13152/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13152/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common Console output https://builds.apache.org/job/PreCommit-YARN-Build/13152/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          curino Carlo Curino added a comment -

          Last few checkstyles (one should be suppressed but it depends on pending YARN-2915 rebasing).

          Show
          curino Carlo Curino added a comment - Last few checkstyles (one should be suppressed but it depends on pending YARN-2915 rebasing).
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
          +1 mvninstall 8m 47s YARN-2915 passed
          +1 compile 0m 22s YARN-2915 passed
          +1 checkstyle 0m 15s YARN-2915 passed
          +1 mvnsite 0m 28s YARN-2915 passed
          +1 mvneclipse 0m 13s YARN-2915 passed
          +1 findbugs 0m 50s YARN-2915 passed
          +1 javadoc 0m 20s YARN-2915 passed
          +1 mvninstall 0m 24s the patch passed
          +1 compile 0m 21s the patch passed
          +1 javac 0m 21s the patch passed
          -1 checkstyle 0m 11s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common: The patch generated 6 new + 0 unchanged - 0 fixed = 6 total (was 0)
          +1 mvnsite 0m 24s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 57s the patch passed
          +1 javadoc 0m 17s the patch passed
          +1 unit 0m 48s hadoop-yarn-server-common in the patch passed.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          16m 6s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828919/YARN-5324-YARN-2915.12.patch
          JIRA Issue YARN-5324
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux edad280b6ed0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision YARN-2915 / b8d9062
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13151/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13151/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13151/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files. +1 mvninstall 8m 47s YARN-2915 passed +1 compile 0m 22s YARN-2915 passed +1 checkstyle 0m 15s YARN-2915 passed +1 mvnsite 0m 28s YARN-2915 passed +1 mvneclipse 0m 13s YARN-2915 passed +1 findbugs 0m 50s YARN-2915 passed +1 javadoc 0m 20s YARN-2915 passed +1 mvninstall 0m 24s the patch passed +1 compile 0m 21s the patch passed +1 javac 0m 21s the patch passed -1 checkstyle 0m 11s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common: The patch generated 6 new + 0 unchanged - 0 fixed = 6 total (was 0) +1 mvnsite 0m 24s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 57s the patch passed +1 javadoc 0m 17s the patch passed +1 unit 0m 48s hadoop-yarn-server-common in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 16m 6s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828919/YARN-5324-YARN-2915.12.patch JIRA Issue YARN-5324 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux edad280b6ed0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision YARN-2915 / b8d9062 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13151/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13151/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common Console output https://builds.apache.org/job/PreCommit-YARN-Build/13151/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          curino Carlo Curino added a comment -

          More massaging for checkstyle and asflicense.

          Show
          curino Carlo Curino added a comment - More massaging for checkstyle and asflicense.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 23s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
          +1 mvninstall 7m 58s YARN-2915 passed
          +1 compile 0m 21s YARN-2915 passed
          +1 checkstyle 0m 15s YARN-2915 passed
          +1 mvnsite 0m 25s YARN-2915 passed
          +1 mvneclipse 0m 13s YARN-2915 passed
          +1 findbugs 0m 48s YARN-2915 passed
          +1 javadoc 0m 17s YARN-2915 passed
          +1 mvninstall 0m 21s the patch passed
          +1 compile 0m 20s the patch passed
          +1 javac 0m 20s the patch passed
          -1 checkstyle 0m 11s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common: The patch generated 18 new + 0 unchanged - 0 fixed = 18 total (was 0)
          +1 mvnsite 0m 23s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 52s the patch passed
          +1 javadoc 0m 14s the patch passed
          +1 unit 0m 44s hadoop-yarn-server-common in the patch passed.
          -1 asflicense 0m 16s The patch generated 1 ASF License warnings.
          14m 54s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828903/YARN-5324-YARN-2915.11.patch
          JIRA Issue YARN-5324
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 387e99511875 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision YARN-2915 / b8d9062
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13128/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13128/testReport/
          asflicense https://builds.apache.org/job/PreCommit-YARN-Build/13128/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13128/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 23s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files. +1 mvninstall 7m 58s YARN-2915 passed +1 compile 0m 21s YARN-2915 passed +1 checkstyle 0m 15s YARN-2915 passed +1 mvnsite 0m 25s YARN-2915 passed +1 mvneclipse 0m 13s YARN-2915 passed +1 findbugs 0m 48s YARN-2915 passed +1 javadoc 0m 17s YARN-2915 passed +1 mvninstall 0m 21s the patch passed +1 compile 0m 20s the patch passed +1 javac 0m 20s the patch passed -1 checkstyle 0m 11s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common: The patch generated 18 new + 0 unchanged - 0 fixed = 18 total (was 0) +1 mvnsite 0m 23s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 52s the patch passed +1 javadoc 0m 14s the patch passed +1 unit 0m 44s hadoop-yarn-server-common in the patch passed. -1 asflicense 0m 16s The patch generated 1 ASF License warnings. 14m 54s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828903/YARN-5324-YARN-2915.11.patch JIRA Issue YARN-5324 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 387e99511875 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision YARN-2915 / b8d9062 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13128/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13128/testReport/ asflicense https://builds.apache.org/job/PreCommit-YARN-Build/13128/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common Console output https://builds.apache.org/job/PreCommit-YARN-Build/13128/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          curino Carlo Curino added a comment - - edited

          In .11. patch, I removed the forward dependency to YARN-5325 from the FederationPoliciesTestUtil class that upset Jenkins.

          Show
          curino Carlo Curino added a comment - - edited In .11. patch, I removed the forward dependency to YARN-5325 from the FederationPoliciesTestUtil class that upset Jenkins.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
          +1 mvninstall 8m 0s YARN-2915 passed
          +1 compile 0m 20s YARN-2915 passed
          +1 checkstyle 0m 13s YARN-2915 passed
          +1 mvnsite 0m 24s YARN-2915 passed
          +1 mvneclipse 0m 13s YARN-2915 passed
          +1 findbugs 0m 43s YARN-2915 passed
          +1 javadoc 0m 16s YARN-2915 passed
          -1 mvninstall 0m 17s hadoop-yarn-server-common in the patch failed.
          -1 compile 0m 18s hadoop-yarn-server-common in the patch failed.
          -1 javac 0m 18s hadoop-yarn-server-common in the patch failed.
          -1 checkstyle 0m 11s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common: The patch generated 18 new + 0 unchanged - 0 fixed = 18 total (was 0)
          -1 mvnsite 0m 18s hadoop-yarn-server-common in the patch failed.
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          -1 findbugs 0m 11s hadoop-yarn-server-common in the patch failed.
          +1 javadoc 0m 14s the patch passed
          -1 unit 0m 17s hadoop-yarn-server-common in the patch failed.
          -1 asflicense 0m 16s The patch generated 1 ASF License warnings.
          13m 16s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828884/YARN-5324-YARN-2915.10.patch
          JIRA Issue YARN-5324
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 6fd95f423bfd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision YARN-2915 / b8d9062
          Default Java 1.8.0_101
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/13127/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          compile https://builds.apache.org/job/PreCommit-YARN-Build/13127/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          javac https://builds.apache.org/job/PreCommit-YARN-Build/13127/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13127/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          mvnsite https://builds.apache.org/job/PreCommit-YARN-Build/13127/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          findbugs https://builds.apache.org/job/PreCommit-YARN-Build/13127/artifact/patchprocess/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/13127/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13127/testReport/
          asflicense https://builds.apache.org/job/PreCommit-YARN-Build/13127/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13127/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files. +1 mvninstall 8m 0s YARN-2915 passed +1 compile 0m 20s YARN-2915 passed +1 checkstyle 0m 13s YARN-2915 passed +1 mvnsite 0m 24s YARN-2915 passed +1 mvneclipse 0m 13s YARN-2915 passed +1 findbugs 0m 43s YARN-2915 passed +1 javadoc 0m 16s YARN-2915 passed -1 mvninstall 0m 17s hadoop-yarn-server-common in the patch failed. -1 compile 0m 18s hadoop-yarn-server-common in the patch failed. -1 javac 0m 18s hadoop-yarn-server-common in the patch failed. -1 checkstyle 0m 11s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common: The patch generated 18 new + 0 unchanged - 0 fixed = 18 total (was 0) -1 mvnsite 0m 18s hadoop-yarn-server-common in the patch failed. +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 0m 11s hadoop-yarn-server-common in the patch failed. +1 javadoc 0m 14s the patch passed -1 unit 0m 17s hadoop-yarn-server-common in the patch failed. -1 asflicense 0m 16s The patch generated 1 ASF License warnings. 13m 16s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828884/YARN-5324-YARN-2915.10.patch JIRA Issue YARN-5324 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 6fd95f423bfd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision YARN-2915 / b8d9062 Default Java 1.8.0_101 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/13127/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt compile https://builds.apache.org/job/PreCommit-YARN-Build/13127/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt javac https://builds.apache.org/job/PreCommit-YARN-Build/13127/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13127/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt mvnsite https://builds.apache.org/job/PreCommit-YARN-Build/13127/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt findbugs https://builds.apache.org/job/PreCommit-YARN-Build/13127/artifact/patchprocess/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13127/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13127/testReport/ asflicense https://builds.apache.org/job/PreCommit-YARN-Build/13127/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common Console output https://builds.apache.org/job/PreCommit-YARN-Build/13127/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          curino Carlo Curino added a comment -

          Subru Krishnan, thanks for the good review. I think I have addressed all your concerns. Below I answer your questions, or explain some of the choices made in addressing your comments.

          Regarding SubClusterIdInfo

          1. Done

          Regarding WeightedPolicyInfo

          1. I addressed all comments with one exception: marshaller and unmarshallers cannot be preinitialized, but JSONJAXBContext can.

          Regarding BaseWeightedRouterPolicy:

          1. the policyInfo check is correct I think (if the current info is not null and is the same as the new one than nothing should be reinitialized)
          2. for a router policy we don't care about what happens for amrmproxy weights.
          3. I moved the checks in a validate method, but I think it is a bit redundant at the moment (the only class that would use it is LoadBasedRaouterPolicy which it doesn't because it would be clumsy).
          4. I don't think adding selectSubCluster changes things much. The general policy API we are implementing is pretty much the same I think.
          5. check for active subclusters is indeed somewhat repeated, though some policies (e.g, UniformRandomRouterPolicy) use it in different ways.

          Regarding LoadBasedRaouterPolicy

          1. I think we should repeat the available memory check every time as this might change based on the latest set of info the FederationStateStoreFacade is returning for the activeSubclusters

          Regarding OrderRouterPolicy

          1. Agreed to rename this to PriorityRouterPolicy. This simple policy is very good for testing (no randomness), and can also provide simple failover type of semantics "I want jobs in this queue to run on subcluster 1, and if that is not available then use subcluster 2, etc.."

          Regarding "Common across tests":

          1. In all/most tests the set of "activeSubclusters" is chosen to be a subset of the one specified in the policy weights. All policies are basically stateless, previous decisions should not affect following ones so the multi invocation tests are only relevant if we check statistical properties (e.g., like we do in testClusterChosenWithRightProbability)
          2. Factored out all common parts, good advice (cut 10kb and increased coverage).
          3. Some of the method in FederationPoliciesTestUtil are used by the upcoming patches for AMRMProxy (I was trying to avoid editing that class over and over at every patch).
          Show
          curino Carlo Curino added a comment - Subru Krishnan , thanks for the good review. I think I have addressed all your concerns. Below I answer your questions, or explain some of the choices made in addressing your comments. Regarding SubClusterIdInfo Done Regarding WeightedPolicyInfo I addressed all comments with one exception: marshaller and unmarshallers cannot be preinitialized, but JSONJAXBContext can. Regarding BaseWeightedRouterPolicy : the policyInfo check is correct I think (if the current info is not null and is the same as the new one than nothing should be reinitialized) for a router policy we don't care about what happens for amrmproxy weights. I moved the checks in a validate method, but I think it is a bit redundant at the moment (the only class that would use it is LoadBasedRaouterPolicy which it doesn't because it would be clumsy). I don't think adding selectSubCluster changes things much. The general policy API we are implementing is pretty much the same I think. check for active subclusters is indeed somewhat repeated, though some policies (e.g, UniformRandomRouterPolicy ) use it in different ways. Regarding LoadBasedRaouterPolicy I think we should repeat the available memory check every time as this might change based on the latest set of info the FederationStateStoreFacade is returning for the activeSubclusters Regarding OrderRouterPolicy Agreed to rename this to PriorityRouterPolicy . This simple policy is very good for testing (no randomness), and can also provide simple failover type of semantics "I want jobs in this queue to run on subcluster 1, and if that is not available then use subcluster 2, etc.." Regarding "Common across tests": In all/most tests the set of "activeSubclusters" is chosen to be a subset of the one specified in the policy weights. All policies are basically stateless, previous decisions should not affect following ones so the multi invocation tests are only relevant if we check statistical properties (e.g., like we do in testClusterChosenWithRightProbability ) Factored out all common parts, good advice (cut 10kb and increased coverage). Some of the method in FederationPoliciesTestUtil are used by the upcoming patches for AMRMProxy (I was trying to avoid editing that class over and over at every patch).
          Hide
          subru Subru Krishnan added a comment -

          Thanks Carlo Curino for the patch. I looked at it and have a few comments below.

          Can we move SubClusterIdInfo --> federation.store.records.dao as it can potentially be used elsewhere like in future Federation Admin REST APIs.

          In WeightedPolicyInfo:

          • Rename routerWeights --> routerPolicyWeights.
          • Rename amrmWeights --> AMRMPolicyWeights
          • Add Javadocs for the getter/setters of the weights as they are not very inituitive.
          • Why are we iterating through the weight maps for every get (getRouterWeights/getAmrmWeights)? Either we should avoid this or do this once at initialization.
          • We should move the JSONJAXBContext and marshaller/unmarshaller as class variables and initialize at declaration as they are expensive ops.
          • Can we give some example for alpha values and their effect to provide more context.
          • To improve readability, can we split the if into two - for routerWeights & amrmWeights respectively;
            if(otherAMRMWeights != null && amrmWeights != null &&
                otherRouterWeights != null && routerWeights != null) {
                 return CollectionUtils.isEqualCollection(otherAMRMWeights.entrySet(),
            	amrmWeights.entrySet()) && CollectionUtils.isEqualCollection(
                      (otherRouterWeights.entrySet()), routerWeights.entrySet());
            
          • For Hashcode -
             31 * amrmWeights + routerWeights 

            should be a better option.

          In BaseWeightedRouterPolicy:

          • Something seems off in the sentence formation of class Javadocs.
          • getRand() seems to be used only by WeightedRandomRouterPolicy so can be moved as a local variable as is done in UniformRandomRouterPolicy.
          • Shouldn't
             if (policyInfo != null && policyInfo.equals(newPolicyInfo)) { 

            be

             if (policyInfo == null || policyInfo.equals(newPolicyInfo)) { 
          • Why are we not checking amrmWeights in:
            if (newWeights == null || newWeights.size() < 1) {
            
          • I think it'll be good if we move all the validations to either a validator or a separate validate method.
          • We should have the check for active sub-clusters here as now every policy repeats it.
          • I feel we should define the protected SubClusterId selectSubCluster(Map<SubClusterId, SubClusterInfo> activeSubclusters, Map<SubClusterId, Float> routerPolicyWeights) in the base class and implementing policies can override it accordingly.

          In LoadBasedRouterPolicy:

          • We should invoke getAvailableMemory only if weight is 1.
          • Do we have to do this for every invocation of getHomeSubcluster. Seems like it but just want to make sure?

          In OrderedRouterPolicy:

          • Typo in Javadoc; Heights should be Highest.
          • I feel it should round robin between the active sub-clusters as current policy will pick the same sub-cluster every time (assuming that entire sub-cluster downtime is rare, more so with RM HA). This feels more like a PriorityRouterPolicy.

          In WeightedRandomRouterPolicy:

          •  if (getPolicyInfo() == null) { 

            isn't the check redundant?

          • Can we add some code comments to clarify how exactly the selection is done.
          •  
            chosen = id;
            if (lookupValue <= 0) {
                break;
            }
            

            should be

             
            if (lookupValue <= 0) {
               return id;
            }
            

          In MockPolicyManager:

          • Might be better to return "default" rather than null in getQueue?

          Common across tests:

          • Am I missing something; All the tests call it out but I don't see scenarios covering multiple invocations or clusters going inactive?
          • We should have a BaseRouterPolicyTest and move the tests there and override only the policy context in individual policy test. Refer to FederationStateStoreBaseTest.
          • How is FederationPoliciesTestUtil::createResourceRequests/createResourceRequest used? Why can't we use ResourceRequest::newInstance for the latter?
          • Nit: variables can be declared outside the loop in setUp

          In TestLoadBasedRouterPolicy:

          • Typo in class Javadoc: weighiting --> weighting
          Show
          subru Subru Krishnan added a comment - Thanks Carlo Curino for the patch. I looked at it and have a few comments below. Can we move SubClusterIdInfo --> federation.store.records.dao as it can potentially be used elsewhere like in future Federation Admin REST APIs. In WeightedPolicyInfo : Rename routerWeights --> routerPolicyWeights . Rename amrmWeights --> AMRMPolicyWeights Add Javadocs for the getter/setters of the weights as they are not very inituitive. Why are we iterating through the weight maps for every get ( getRouterWeights/getAmrmWeights )? Either we should avoid this or do this once at initialization. We should move the JSONJAXBContext and marshaller/unmarshaller as class variables and initialize at declaration as they are expensive ops. Can we give some example for alpha values and their effect to provide more context. To improve readability, can we split the if into two - for routerWeights & amrmWeights respectively; if (otherAMRMWeights != null && amrmWeights != null && otherRouterWeights != null && routerWeights != null ) { return CollectionUtils.isEqualCollection(otherAMRMWeights.entrySet(), amrmWeights.entrySet()) && CollectionUtils.isEqualCollection( (otherRouterWeights.entrySet()), routerWeights.entrySet()); For Hashcode - 31 * amrmWeights + routerWeights should be a better option. In BaseWeightedRouterPolicy : Something seems off in the sentence formation of class Javadocs. getRand() seems to be used only by WeightedRandomRouterPolicy so can be moved as a local variable as is done in UniformRandomRouterPolicy . Shouldn't if (policyInfo != null && policyInfo.equals(newPolicyInfo)) { be if (policyInfo == null || policyInfo.equals(newPolicyInfo)) { Why are we not checking amrmWeights in: if (newWeights == null || newWeights.size() < 1) { I think it'll be good if we move all the validations to either a validator or a separate validate method. We should have the check for active sub-clusters here as now every policy repeats it. I feel we should define the protected SubClusterId selectSubCluster(Map<SubClusterId, SubClusterInfo> activeSubclusters, Map<SubClusterId, Float> routerPolicyWeights) in the base class and implementing policies can override it accordingly. In LoadBasedRouterPolicy : We should invoke getAvailableMemory only if weight is 1. Do we have to do this for every invocation of getHomeSubcluster . Seems like it but just want to make sure? In OrderedRouterPolicy : Typo in Javadoc; Heights should be Highest . I feel it should round robin between the active sub-clusters as current policy will pick the same sub-cluster every time (assuming that entire sub-cluster downtime is rare, more so with RM HA). This feels more like a PriorityRouterPolicy . In WeightedRandomRouterPolicy : if (getPolicyInfo() == null ) { isn't the check redundant? Can we add some code comments to clarify how exactly the selection is done. chosen = id; if (lookupValue <= 0) { break ; } should be if (lookupValue <= 0) { return id; } In MockPolicyManager : Might be better to return "default" rather than null in getQueue ? Common across tests: Am I missing something; All the tests call it out but I don't see scenarios covering multiple invocations or clusters going inactive? We should have a BaseRouterPolicyTest and move the tests there and override only the policy context in individual policy test. Refer to FederationStateStoreBaseTest . How is FederationPoliciesTestUtil::createResourceRequests/createResourceRequest used? Why can't we use ResourceRequest::newInstance for the latter? Nit: variables can be declared outside the loop in setUp In TestLoadBasedRouterPolicy : Typo in class Javadoc: weighiting --> weighting
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
          +1 mvninstall 7m 39s YARN-2915 passed
          +1 compile 0m 22s YARN-2915 passed
          +1 checkstyle 0m 14s YARN-2915 passed
          +1 mvnsite 0m 26s YARN-2915 passed
          +1 mvneclipse 0m 13s YARN-2915 passed
          +1 findbugs 0m 45s YARN-2915 passed
          +1 javadoc 0m 17s YARN-2915 passed
          +1 mvninstall 0m 20s the patch passed
          +1 compile 0m 18s the patch passed
          +1 javac 0m 18s the patch passed
          -1 checkstyle 0m 11s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
          +1 mvnsite 0m 22s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 51s the patch passed
          +1 javadoc 0m 15s the patch passed
          +1 unit 0m 53s hadoop-yarn-server-common in the patch passed.
          +1 asflicense 0m 16s The patch does not generate ASF License warnings.
          14m 34s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828108/YARN-5324-YARN-2915.09.patch
          JIRA Issue YARN-5324
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 411c7df3a9a1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision YARN-2915 / 302d206
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13091/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13091/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13091/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files. +1 mvninstall 7m 39s YARN-2915 passed +1 compile 0m 22s YARN-2915 passed +1 checkstyle 0m 14s YARN-2915 passed +1 mvnsite 0m 26s YARN-2915 passed +1 mvneclipse 0m 13s YARN-2915 passed +1 findbugs 0m 45s YARN-2915 passed +1 javadoc 0m 17s YARN-2915 passed +1 mvninstall 0m 20s the patch passed +1 compile 0m 18s the patch passed +1 javac 0m 18s the patch passed -1 checkstyle 0m 11s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) +1 mvnsite 0m 22s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 51s the patch passed +1 javadoc 0m 15s the patch passed +1 unit 0m 53s hadoop-yarn-server-common in the patch passed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 14m 34s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828108/YARN-5324-YARN-2915.09.patch JIRA Issue YARN-5324 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 411c7df3a9a1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision YARN-2915 / 302d206 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13091/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13091/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common Console output https://builds.apache.org/job/PreCommit-YARN-Build/13091/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          curino Carlo Curino added a comment -

          More cleanups. The remaining checkstyle (>7 params) I think is ok as is (used only by tests to create a ResourceRequest for testing).

          Show
          curino Carlo Curino added a comment - More cleanups. The remaining checkstyle (>7 params) I think is ok as is (used only by tests to create a ResourceRequest for testing).
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
          +1 mvninstall 9m 6s YARN-2915 passed
          +1 compile 0m 21s YARN-2915 passed
          +1 checkstyle 0m 14s YARN-2915 passed
          +1 mvnsite 0m 24s YARN-2915 passed
          +1 mvneclipse 0m 12s YARN-2915 passed
          +1 findbugs 0m 45s YARN-2915 passed
          +1 javadoc 0m 17s YARN-2915 passed
          +1 mvninstall 0m 19s the patch passed
          +1 compile 0m 18s the patch passed
          +1 javac 0m 18s the patch passed
          -1 checkstyle 0m 11s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common: The patch generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0)
          +1 mvnsite 0m 22s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          -1 findbugs 0m 52s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
          +1 javadoc 0m 15s the patch passed
          +1 unit 0m 51s hadoop-yarn-server-common in the patch passed.
          +1 asflicense 0m 15s The patch does not generate ASF License warnings.
          15m 51s



          Reason Tests
          FindBugs module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
            org.apache.hadoop.yarn.server.federation.policies.dao.WeightedPolicyInfo defines equals and uses Object.hashCode() At WeightedPolicyInfo.java:Object.hashCode() At WeightedPolicyInfo.java:[lines 140-157]



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828092/YARN-5324-YARN-2915.08.patch
          JIRA Issue YARN-5324
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 3d93d9d04bd3 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision YARN-2915 / 302d206
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13090/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          findbugs https://builds.apache.org/job/PreCommit-YARN-Build/13090/artifact/patchprocess/new-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.html
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13090/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13090/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files. +1 mvninstall 9m 6s YARN-2915 passed +1 compile 0m 21s YARN-2915 passed +1 checkstyle 0m 14s YARN-2915 passed +1 mvnsite 0m 24s YARN-2915 passed +1 mvneclipse 0m 12s YARN-2915 passed +1 findbugs 0m 45s YARN-2915 passed +1 javadoc 0m 17s YARN-2915 passed +1 mvninstall 0m 19s the patch passed +1 compile 0m 18s the patch passed +1 javac 0m 18s the patch passed -1 checkstyle 0m 11s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common: The patch generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0) +1 mvnsite 0m 22s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 0m 52s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) +1 javadoc 0m 15s the patch passed +1 unit 0m 51s hadoop-yarn-server-common in the patch passed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 15m 51s Reason Tests FindBugs module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common   org.apache.hadoop.yarn.server.federation.policies.dao.WeightedPolicyInfo defines equals and uses Object.hashCode() At WeightedPolicyInfo.java:Object.hashCode() At WeightedPolicyInfo.java: [lines 140-157] Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828092/YARN-5324-YARN-2915.08.patch JIRA Issue YARN-5324 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 3d93d9d04bd3 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision YARN-2915 / 302d206 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13090/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt findbugs https://builds.apache.org/job/PreCommit-YARN-Build/13090/artifact/patchprocess/new-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.html Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13090/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common Console output https://builds.apache.org/job/PreCommit-YARN-Build/13090/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          curino Carlo Curino added a comment -

          Standard pass on fixing checkstyle/javadoc/findbugs/asflicense issues.

          Show
          curino Carlo Curino added a comment - Standard pass on fixing checkstyle/javadoc/findbugs/asflicense issues.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
          +1 mvninstall 7m 9s YARN-2915 passed
          +1 compile 0m 20s YARN-2915 passed
          +1 checkstyle 0m 13s YARN-2915 passed
          +1 mvnsite 0m 24s YARN-2915 passed
          +1 mvneclipse 0m 13s YARN-2915 passed
          +1 findbugs 0m 43s YARN-2915 passed
          +1 javadoc 0m 16s YARN-2915 passed
          +1 mvninstall 0m 18s the patch passed
          +1 compile 0m 18s the patch passed
          -1 javac 0m 18s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
          -1 checkstyle 0m 11s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common: The patch generated 67 new + 0 unchanged - 0 fixed = 67 total (was 0)
          +1 mvnsite 0m 20s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          -1 findbugs 0m 49s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common generated 4 new + 0 unchanged - 0 fixed = 4 total (was 0)
          -1 javadoc 0m 13s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common generated 2 new + 160 unchanged - 0 fixed = 162 total (was 160)
          +1 unit 0m 49s hadoop-yarn-server-common in the patch passed.
          -1 asflicense 0m 16s The patch generated 2 ASF License warnings.
          13m 42s



          Reason Tests
          FindBugs module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
            Found reliance on default encoding in org.apache.hadoop.yarn.server.federation.policies.dao.WeightedPolicyInfo.fromByteBuffer(ByteBuffer):in org.apache.hadoop.yarn.server.federation.policies.dao.WeightedPolicyInfo.fromByteBuffer(ByteBuffer): new String(byte[]) At WeightedPolicyInfo.java:[line 101]
            Found reliance on default encoding in org.apache.hadoop.yarn.server.federation.policies.dao.WeightedPolicyInfo.toByteBuffer(WeightedPolicyInfo):in org.apache.hadoop.yarn.server.federation.policies.dao.WeightedPolicyInfo.toByteBuffer(WeightedPolicyInfo): String.getBytes() At WeightedPolicyInfo.java:[line 116]
            org.apache.hadoop.yarn.server.federation.policies.dao.WeightedPolicyInfo defines equals and uses Object.hashCode() At WeightedPolicyInfo.java:Object.hashCode() At WeightedPolicyInfo.java:[lines 136-147]
            org.apache.hadoop.yarn.server.federation.policies.dao.WeightedPolicyInfo.equals(Object) does not check for null argument At WeightedPolicyInfo.java:null argument At WeightedPolicyInfo.java:[lines 136-147]



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12827865/YARN-5324-YARN-2915.07.patch
          JIRA Issue YARN-5324
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 73b3e89e5c53 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision YARN-2915 / 302d206
          Default Java 1.8.0_101
          findbugs v3.0.0
          javac https://builds.apache.org/job/PreCommit-YARN-Build/13068/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13068/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          findbugs https://builds.apache.org/job/PreCommit-YARN-Build/13068/artifact/patchprocess/new-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.html
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13068/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13068/testReport/
          asflicense https://builds.apache.org/job/PreCommit-YARN-Build/13068/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13068/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files. +1 mvninstall 7m 9s YARN-2915 passed +1 compile 0m 20s YARN-2915 passed +1 checkstyle 0m 13s YARN-2915 passed +1 mvnsite 0m 24s YARN-2915 passed +1 mvneclipse 0m 13s YARN-2915 passed +1 findbugs 0m 43s YARN-2915 passed +1 javadoc 0m 16s YARN-2915 passed +1 mvninstall 0m 18s the patch passed +1 compile 0m 18s the patch passed -1 javac 0m 18s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) -1 checkstyle 0m 11s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common: The patch generated 67 new + 0 unchanged - 0 fixed = 67 total (was 0) +1 mvnsite 0m 20s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 0m 49s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common generated 4 new + 0 unchanged - 0 fixed = 4 total (was 0) -1 javadoc 0m 13s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common generated 2 new + 160 unchanged - 0 fixed = 162 total (was 160) +1 unit 0m 49s hadoop-yarn-server-common in the patch passed. -1 asflicense 0m 16s The patch generated 2 ASF License warnings. 13m 42s Reason Tests FindBugs module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common   Found reliance on default encoding in org.apache.hadoop.yarn.server.federation.policies.dao.WeightedPolicyInfo.fromByteBuffer(ByteBuffer):in org.apache.hadoop.yarn.server.federation.policies.dao.WeightedPolicyInfo.fromByteBuffer(ByteBuffer): new String(byte[]) At WeightedPolicyInfo.java: [line 101]   Found reliance on default encoding in org.apache.hadoop.yarn.server.federation.policies.dao.WeightedPolicyInfo.toByteBuffer(WeightedPolicyInfo):in org.apache.hadoop.yarn.server.federation.policies.dao.WeightedPolicyInfo.toByteBuffer(WeightedPolicyInfo): String.getBytes() At WeightedPolicyInfo.java: [line 116]   org.apache.hadoop.yarn.server.federation.policies.dao.WeightedPolicyInfo defines equals and uses Object.hashCode() At WeightedPolicyInfo.java:Object.hashCode() At WeightedPolicyInfo.java: [lines 136-147]   org.apache.hadoop.yarn.server.federation.policies.dao.WeightedPolicyInfo.equals(Object) does not check for null argument At WeightedPolicyInfo.java:null argument At WeightedPolicyInfo.java: [lines 136-147] Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12827865/YARN-5324-YARN-2915.07.patch JIRA Issue YARN-5324 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 73b3e89e5c53 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision YARN-2915 / 302d206 Default Java 1.8.0_101 findbugs v3.0.0 javac https://builds.apache.org/job/PreCommit-YARN-Build/13068/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13068/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt findbugs https://builds.apache.org/job/PreCommit-YARN-Build/13068/artifact/patchprocess/new-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.html javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13068/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13068/testReport/ asflicense https://builds.apache.org/job/PreCommit-YARN-Build/13068/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common Console output https://builds.apache.org/job/PreCommit-YARN-Build/13068/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          curino Carlo Curino added a comment -

          Rebased the patch based on YARN-5323.

          The patch does a minor updated of the APIs in YARN-5323:

          1. merges FederationPolicyWriter and FederationPolicyConfigurator in a FederationPolicyManager. This simplifies class structure and allows for class for name instantiations more easily.
          2. renames a method in setFederationPolicyConfiguration to setSubClusterPolicyConfiguration (missed in previous refactoring)

          The patch also provides 4 implementations for the RouterFederationPolicy, and general DAO and utils classes (which we will also use in YARN-5325 and YARN-5391).

          Subru Krishnan, Giovanni Matteo Fumarola and Tan, Wangda I think this should be ready for review.

          Show
          curino Carlo Curino added a comment - Rebased the patch based on YARN-5323 . The patch does a minor updated of the APIs in YARN-5323 : merges FederationPolicyWriter and FederationPolicyConfigurator in a FederationPolicyManager . This simplifies class structure and allows for class for name instantiations more easily. renames a method in setFederationPolicyConfiguration to setSubClusterPolicyConfiguration (missed in previous refactoring) The patch also provides 4 implementations for the RouterFederationPolicy , and general DAO and utils classes (which we will also use in YARN-5325 and YARN-5391 ). Subru Krishnan , Giovanni Matteo Fumarola and Tan, Wangda I think this should be ready for review.
          Hide
          curino Carlo Curino added a comment -

          Much of the errors are because this depends on YARN-5323

          Show
          curino Carlo Curino added a comment - Much of the errors are because this depends on YARN-5323
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          +1 mvninstall 10m 34s YARN-2915 passed
          +1 compile 0m 20s YARN-2915 passed
          +1 checkstyle 0m 15s YARN-2915 passed
          +1 mvnsite 0m 24s YARN-2915 passed
          +1 mvneclipse 0m 19s YARN-2915 passed
          +1 findbugs 0m 51s YARN-2915 passed
          +1 javadoc 0m 16s YARN-2915 passed
          -1 mvninstall 0m 14s hadoop-yarn-server-common in the patch failed.
          -1 compile 0m 14s hadoop-yarn-server-common in the patch failed.
          -1 javac 0m 14s hadoop-yarn-server-common in the patch failed.
          -1 checkstyle 0m 11s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common: The patch generated 63 new + 0 unchanged - 0 fixed = 63 total (was 0)
          -1 mvnsite 0m 15s hadoop-yarn-server-common in the patch failed.
          +1 mvneclipse 0m 9s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          -1 findbugs 0m 14s hadoop-yarn-server-common in the patch failed.
          -1 javadoc 0m 13s hadoop-yarn-server-common in the patch failed.
          -1 unit 0m 14s hadoop-yarn-server-common in the patch failed.
          -1 asflicense 0m 17s The patch generated 1 ASF License warnings.
          15m 51s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12823157/YARN-5324-YARN-2915.06.patch
          JIRA Issue YARN-5324
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 56ea4710e01f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision YARN-2915 / b689f55
          Default Java 1.8.0_101
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/12734/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          compile https://builds.apache.org/job/PreCommit-YARN-Build/12734/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          javac https://builds.apache.org/job/PreCommit-YARN-Build/12734/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12734/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          mvnsite https://builds.apache.org/job/PreCommit-YARN-Build/12734/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          findbugs https://builds.apache.org/job/PreCommit-YARN-Build/12734/artifact/patchprocess/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12734/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/12734/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12734/testReport/
          asflicense https://builds.apache.org/job/PreCommit-YARN-Build/12734/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12734/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. +1 mvninstall 10m 34s YARN-2915 passed +1 compile 0m 20s YARN-2915 passed +1 checkstyle 0m 15s YARN-2915 passed +1 mvnsite 0m 24s YARN-2915 passed +1 mvneclipse 0m 19s YARN-2915 passed +1 findbugs 0m 51s YARN-2915 passed +1 javadoc 0m 16s YARN-2915 passed -1 mvninstall 0m 14s hadoop-yarn-server-common in the patch failed. -1 compile 0m 14s hadoop-yarn-server-common in the patch failed. -1 javac 0m 14s hadoop-yarn-server-common in the patch failed. -1 checkstyle 0m 11s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common: The patch generated 63 new + 0 unchanged - 0 fixed = 63 total (was 0) -1 mvnsite 0m 15s hadoop-yarn-server-common in the patch failed. +1 mvneclipse 0m 9s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 0m 14s hadoop-yarn-server-common in the patch failed. -1 javadoc 0m 13s hadoop-yarn-server-common in the patch failed. -1 unit 0m 14s hadoop-yarn-server-common in the patch failed. -1 asflicense 0m 17s The patch generated 1 ASF License warnings. 15m 51s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12823157/YARN-5324-YARN-2915.06.patch JIRA Issue YARN-5324 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 56ea4710e01f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision YARN-2915 / b689f55 Default Java 1.8.0_101 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/12734/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt compile https://builds.apache.org/job/PreCommit-YARN-Build/12734/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt javac https://builds.apache.org/job/PreCommit-YARN-Build/12734/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12734/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt mvnsite https://builds.apache.org/job/PreCommit-YARN-Build/12734/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt findbugs https://builds.apache.org/job/PreCommit-YARN-Build/12734/artifact/patchprocess/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12734/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12734/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12734/testReport/ asflicense https://builds.apache.org/job/PreCommit-YARN-Build/12734/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common Console output https://builds.apache.org/job/PreCommit-YARN-Build/12734/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          +1 mvninstall 7m 14s trunk passed
          +1 compile 0m 18s trunk passed
          +1 checkstyle 0m 12s trunk passed
          +1 mvnsite 0m 23s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 0m 40s trunk passed
          +1 javadoc 0m 16s trunk passed
          -1 mvninstall 0m 15s hadoop-yarn-server-common in the patch failed.
          -1 compile 0m 15s hadoop-yarn-server-common in the patch failed.
          -1 javac 0m 15s hadoop-yarn-server-common in the patch failed.
          -1 checkstyle 0m 11s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common: The patch generated 63 new + 0 unchanged - 0 fixed = 63 total (was 0)
          -1 mvnsite 0m 15s hadoop-yarn-server-common in the patch failed.
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          -1 findbugs 0m 15s hadoop-yarn-server-common in the patch failed.
          -1 javadoc 0m 13s hadoop-yarn-server-common in the patch failed.
          -1 unit 0m 14s hadoop-yarn-server-common in the patch failed.
          -1 asflicense 0m 19s The patch generated 1 ASF License warnings.
          12m 15s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12823088/YARN-5324.05.patch
          JIRA Issue YARN-5324
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux b19ea111023d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / e83be44
          Default Java 1.8.0_101
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/12730/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          compile https://builds.apache.org/job/PreCommit-YARN-Build/12730/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          javac https://builds.apache.org/job/PreCommit-YARN-Build/12730/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12730/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          mvnsite https://builds.apache.org/job/PreCommit-YARN-Build/12730/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          findbugs https://builds.apache.org/job/PreCommit-YARN-Build/12730/artifact/patchprocess/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12730/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/12730/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12730/testReport/
          asflicense https://builds.apache.org/job/PreCommit-YARN-Build/12730/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12730/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. +1 mvninstall 7m 14s trunk passed +1 compile 0m 18s trunk passed +1 checkstyle 0m 12s trunk passed +1 mvnsite 0m 23s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 0m 40s trunk passed +1 javadoc 0m 16s trunk passed -1 mvninstall 0m 15s hadoop-yarn-server-common in the patch failed. -1 compile 0m 15s hadoop-yarn-server-common in the patch failed. -1 javac 0m 15s hadoop-yarn-server-common in the patch failed. -1 checkstyle 0m 11s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common: The patch generated 63 new + 0 unchanged - 0 fixed = 63 total (was 0) -1 mvnsite 0m 15s hadoop-yarn-server-common in the patch failed. +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 0m 15s hadoop-yarn-server-common in the patch failed. -1 javadoc 0m 13s hadoop-yarn-server-common in the patch failed. -1 unit 0m 14s hadoop-yarn-server-common in the patch failed. -1 asflicense 0m 19s The patch generated 1 ASF License warnings. 12m 15s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12823088/YARN-5324.05.patch JIRA Issue YARN-5324 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b19ea111023d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e83be44 Default Java 1.8.0_101 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/12730/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt compile https://builds.apache.org/job/PreCommit-YARN-Build/12730/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt javac https://builds.apache.org/job/PreCommit-YARN-Build/12730/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12730/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt mvnsite https://builds.apache.org/job/PreCommit-YARN-Build/12730/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt findbugs https://builds.apache.org/job/PreCommit-YARN-Build/12730/artifact/patchprocess/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12730/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12730/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12730/testReport/ asflicense https://builds.apache.org/job/PreCommit-YARN-Build/12730/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common Console output https://builds.apache.org/job/PreCommit-YARN-Build/12730/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          curino Carlo Curino added a comment -

          (renaming to target branch for Jenkins)

          Show
          curino Carlo Curino added a comment - (renaming to target branch for Jenkins)
          Hide
          curino Carlo Curino added a comment -

          Rebased after YARN-5390 is committed/evolved

          Show
          curino Carlo Curino added a comment - Rebased after YARN-5390 is committed/evolved
          Hide
          curino Carlo Curino added a comment -

          Young Chen thanks for reviewing this.

          1. The ProbabilisticRouterFederationPolicy was renamed to WeightedRandomRouterFederationPolicy.
          2. The snippet you quote was still there, good catch. I added to all test cases a corner check that forces us to be aware of weights missing info about an active clusters, and fixed the NPE in WeightedRandomRouterFederationPolicy and other classes.

          Also:

          1. The latest patch also includes a LoadBasedRouterFederationPolicy that parses the ClusterMetricsInfo from String capability and picks the cluster with the most free space max(availableMemoryMB (and adds tests for that and OrderedRouterFederationPolicy.
          2. We could probably polish the tests (I think there is a bunch of redundancy.
          Show
          curino Carlo Curino added a comment - Young Chen thanks for reviewing this. The ProbabilisticRouterFederationPolicy was renamed to WeightedRandomRouterFederationPolicy . The snippet you quote was still there, good catch. I added to all test cases a corner check that forces us to be aware of weights missing info about an active clusters, and fixed the NPE in WeightedRandomRouterFederationPolicy and other classes. Also: The latest patch also includes a LoadBasedRouterFederationPolicy that parses the ClusterMetricsInfo from String capability and picks the cluster with the most free space max(availableMemoryMB (and adds tests for that and OrderedRouterFederationPolicy . We could probably polish the tests (I think there is a bunch of redundancy.
          Hide
          yochen Young Chen added a comment -

          Thanks Carlo Curino,

          Just a couple things I noticed when going through the patch:

          • The second patch is missing the ProbabilisticRouterFederationPolicy, is this intentional?
          • In the first patch, in ProbabilisticRouterFederationPolicy there's this section of code
                        float totActiveWeight = 0;
            	    for (FederationSubClusterId id : activeSubclusters.keySet()) {
            	      totActiveWeight += weights.get(id);
            	    }
            

            If weights does not include all the sub clusters, or a newly active sub cluster shows up in activeSubClusters, you can potentially have a nullptrexception here.

          Show
          yochen Young Chen added a comment - Thanks Carlo Curino , Just a couple things I noticed when going through the patch: The second patch is missing the ProbabilisticRouterFederationPolicy, is this intentional? In the first patch, in ProbabilisticRouterFederationPolicy there's this section of code float totActiveWeight = 0; for (FederationSubClusterId id : activeSubclusters.keySet()) { totActiveWeight += weights.get(id); } If weights does not include all the sub clusters, or a newly active sub cluster shows up in activeSubClusters, you can potentially have a nullptrexception here.
          Hide
          curino Carlo Curino added a comment -

          This patch provides 3 simple implementation of RouterFederationPolicy:

          1. A uniform random choice among active subclusters
          2. A weighted random choice among active subclusters, where the weights in the policy determine likelihood of picking a cluster (and =0 means actively excluding a subcluster)
          3. A ordered priority among active subclusters, where the weights in the policy determine the "cost" of picking a cluster. The policy always picks the subcluster with the lowest weight.
          Show
          curino Carlo Curino added a comment - This patch provides 3 simple implementation of RouterFederationPolicy : A uniform random choice among active subclusters A weighted random choice among active subclusters, where the weights in the policy determine likelihood of picking a cluster (and =0 means actively excluding a subcluster) A ordered priority among active subclusters, where the weights in the policy determine the "cost" of picking a cluster. The policy always picks the subcluster with the lowest weight.

            People

            • Assignee:
              curino Carlo Curino
              Reporter:
              curino Carlo Curino
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development