Details

    • Reviewed

    Attachments

      1. YARN-8973.v3.patch
        18 kB
        Giovanni Matteo Fumarola
      2. YARN-8973.v2.patch
        19 kB
        Giovanni Matteo Fumarola
      3. YARN-8973.v1.patch
        14 kB
        Giovanni Matteo Fumarola

      Issue Links

        Activity

          elgoiri Íñigo Goiri added a comment -

          Thanks giovanni.fumarola for YARN-8973.v1.patch.

          • Where do the new methods come from? Is there any interface we can leverage?
          • Avoid changing FederationInterceptorREST line 1286.
          • Any tests?
          elgoiri Íñigo Goiri added a comment - Thanks giovanni.fumarola for YARN-8973.v1.patch . Where do the new methods come from? Is there any interface we can leverage? Avoid changing FederationInterceptorREST line 1286. Any tests?
          giovanni.fumarola Giovanni Matteo Fumarola added a comment - - edited

          Thanks elgoiri.

          • YARN-8559 and YARN-5952 added 2 REST methods in RMWebService without inserting them in the RMWebServiceProtocol. The jira tracks the effort make sure everything is in place.
          • I will remove.
          • Let me add in the Router.
          giovanni.fumarola Giovanni Matteo Fumarola added a comment - - edited Thanks elgoiri . YARN-8559 and YARN-5952 added 2 REST methods in RMWebService without inserting them in the RMWebServiceProtocol. The jira tracks the effort make sure everything is in place. I will remove. Let me add in the Router.
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
                trunk Compile Tests
          0 mvndep 0m 11s Maven dependency ordering for branch
          +1 mvninstall 20m 26s trunk passed
          +1 compile 2m 56s trunk passed
          +1 checkstyle 1m 1s trunk passed
          +1 mvnsite 1m 19s trunk passed
          +1 shadedclient 14m 44s branch has no errors when building and testing our client artifacts.
          +1 findbugs 2m 5s trunk passed
          +1 javadoc 0m 51s trunk passed
                Patch Compile Tests
          0 mvndep 0m 12s Maven dependency ordering for patch
          +1 mvninstall 1m 23s the patch passed
          +1 compile 2m 49s the patch passed
          +1 javac 2m 49s the patch passed
          +1 checkstyle 1m 1s the patch passed
          +1 mvnsite 1m 16s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 shadedclient 13m 19s patch has no errors when building and testing our client artifacts.
          +1 findbugs 2m 3s the patch passed
          +1 javadoc 0m 46s the patch passed
                Other Tests
          -1 unit 108m 31s hadoop-yarn-server-resourcemanager in the patch failed.
          +1 unit 1m 51s hadoop-yarn-server-router in the patch passed.
          +1 asflicense 0m 25s The patch does not generate ASF License warnings.
          176m 41s



          Reason Tests
          Failed junit tests hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy



          Subsystem Report/Notes
          Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f
          JIRA Issue YARN-8973
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12946967/YARN-8973.v1.patch
          Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
          uname Linux 9a44925f8982 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/patchprocess/precommit/personality/provided.sh
          git revision trunk / f3f5e7a
          maven version: Apache Maven 3.3.9
          Default Java 1.8.0_181
          findbugs v3.1.0-RC1
          unit https://builds.apache.org/job/PreCommit-YARN-Build/22419/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/22419/testReport/
          Max. process+thread count 869 (vs. ulimit of 10000)
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/22419/console
          Powered by Apache Yetus 0.8.0 http://yetus.apache.org

          This message was automatically generated.

          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.       trunk Compile Tests 0 mvndep 0m 11s Maven dependency ordering for branch +1 mvninstall 20m 26s trunk passed +1 compile 2m 56s trunk passed +1 checkstyle 1m 1s trunk passed +1 mvnsite 1m 19s trunk passed +1 shadedclient 14m 44s branch has no errors when building and testing our client artifacts. +1 findbugs 2m 5s trunk passed +1 javadoc 0m 51s trunk passed       Patch Compile Tests 0 mvndep 0m 12s Maven dependency ordering for patch +1 mvninstall 1m 23s the patch passed +1 compile 2m 49s the patch passed +1 javac 2m 49s the patch passed +1 checkstyle 1m 1s the patch passed +1 mvnsite 1m 16s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedclient 13m 19s patch has no errors when building and testing our client artifacts. +1 findbugs 2m 3s the patch passed +1 javadoc 0m 46s the patch passed       Other Tests -1 unit 108m 31s hadoop-yarn-server-resourcemanager in the patch failed. +1 unit 1m 51s hadoop-yarn-server-router in the patch passed. +1 asflicense 0m 25s The patch does not generate ASF License warnings. 176m 41s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f JIRA Issue YARN-8973 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12946967/YARN-8973.v1.patch Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle uname Linux 9a44925f8982 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision trunk / f3f5e7a maven version: Apache Maven 3.3.9 Default Java 1.8.0_181 findbugs v3.1.0-RC1 unit https://builds.apache.org/job/PreCommit-YARN-Build/22419/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/22419/testReport/ Max. process+thread count 869 (vs. ulimit of 10000) modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server Console output https://builds.apache.org/job/PreCommit-YARN-Build/22419/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
                trunk Compile Tests
          0 mvndep 0m 11s Maven dependency ordering for branch
          +1 mvninstall 19m 53s trunk passed
          +1 compile 2m 37s trunk passed
          +1 checkstyle 0m 58s trunk passed
          +1 mvnsite 1m 15s trunk passed
          +1 shadedclient 14m 17s branch has no errors when building and testing our client artifacts.
          +1 findbugs 1m 46s trunk passed
          +1 javadoc 0m 49s trunk passed
                Patch Compile Tests
          0 mvndep 0m 11s Maven dependency ordering for patch
          +1 mvninstall 1m 12s the patch passed
          +1 compile 2m 31s the patch passed
          +1 javac 2m 31s the patch passed
          -0 checkstyle 0m 54s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 1 new + 18 unchanged - 0 fixed = 19 total (was 18)
          +1 mvnsite 1m 8s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 shadedclient 13m 52s patch has no errors when building and testing our client artifacts.
          +1 findbugs 2m 16s the patch passed
          +1 javadoc 0m 50s the patch passed
                Other Tests
          -1 unit 105m 12s hadoop-yarn-server-resourcemanager in the patch failed.
          +1 unit 1m 32s hadoop-yarn-server-router in the patch passed.
          +1 asflicense 0m 26s The patch does not generate ASF License warnings.
          171m 27s



          Subsystem Report/Notes
          Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f
          JIRA Issue YARN-8973
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12947118/YARN-8973.v2.patch
          Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
          uname Linux 09e7b6f5b33f 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/patchprocess/precommit/personality/provided.sh
          git revision trunk / 00a67f7
          maven version: Apache Maven 3.3.9
          Default Java 1.8.0_181
          findbugs v3.1.0-RC1
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/22434/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/22434/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/22434/testReport/
          Max. process+thread count 962 (vs. ulimit of 10000)
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/22434/console
          Powered by Apache Yetus 0.8.0 http://yetus.apache.org

          This message was automatically generated.

          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.       trunk Compile Tests 0 mvndep 0m 11s Maven dependency ordering for branch +1 mvninstall 19m 53s trunk passed +1 compile 2m 37s trunk passed +1 checkstyle 0m 58s trunk passed +1 mvnsite 1m 15s trunk passed +1 shadedclient 14m 17s branch has no errors when building and testing our client artifacts. +1 findbugs 1m 46s trunk passed +1 javadoc 0m 49s trunk passed       Patch Compile Tests 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 1m 12s the patch passed +1 compile 2m 31s the patch passed +1 javac 2m 31s the patch passed -0 checkstyle 0m 54s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 1 new + 18 unchanged - 0 fixed = 19 total (was 18) +1 mvnsite 1m 8s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedclient 13m 52s patch has no errors when building and testing our client artifacts. +1 findbugs 2m 16s the patch passed +1 javadoc 0m 50s the patch passed       Other Tests -1 unit 105m 12s hadoop-yarn-server-resourcemanager in the patch failed. +1 unit 1m 32s hadoop-yarn-server-router in the patch passed. +1 asflicense 0m 26s The patch does not generate ASF License warnings. 171m 27s Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f JIRA Issue YARN-8973 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12947118/YARN-8973.v2.patch Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle uname Linux 09e7b6f5b33f 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision trunk / 00a67f7 maven version: Apache Maven 3.3.9 Default Java 1.8.0_181 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/22434/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/22434/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/22434/testReport/ Max. process+thread count 962 (vs. ulimit of 10000) modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server Console output https://builds.apache.org/job/PreCommit-YARN-Build/22434/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
                trunk Compile Tests
          0 mvndep 0m 11s Maven dependency ordering for branch
          +1 mvninstall 20m 9s trunk passed
          +1 compile 2m 47s trunk passed
          +1 checkstyle 1m 1s trunk passed
          +1 mvnsite 1m 16s trunk passed
          +1 shadedclient 14m 45s branch has no errors when building and testing our client artifacts.
          +1 findbugs 1m 57s trunk passed
          +1 javadoc 0m 49s trunk passed
                Patch Compile Tests
          0 mvndep 0m 11s Maven dependency ordering for patch
          +1 mvninstall 1m 14s the patch passed
          +1 compile 2m 40s the patch passed
          +1 javac 2m 40s the patch passed
          +1 checkstyle 0m 54s the patch passed
          +1 mvnsite 1m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 shadedclient 13m 49s patch has no errors when building and testing our client artifacts.
          +1 findbugs 2m 7s the patch passed
          +1 javadoc 0m 49s the patch passed
                Other Tests
          -1 unit 108m 17s hadoop-yarn-server-resourcemanager in the patch failed.
          +1 unit 1m 48s hadoop-yarn-server-router in the patch passed.
          +1 asflicense 0m 25s The patch does not generate ASF License warnings.
          175m 51s



          Reason Tests
          Failed junit tests hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy



          Subsystem Report/Notes
          Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f
          JIRA Issue YARN-8973
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12947135/YARN-8973.v3.patch
          Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
          uname Linux e354aad4ccb6 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/patchprocess/precommit/personality/provided.sh
          git revision trunk / 08d69d9
          maven version: Apache Maven 3.3.9
          Default Java 1.8.0_181
          findbugs v3.1.0-RC1
          unit https://builds.apache.org/job/PreCommit-YARN-Build/22436/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/22436/testReport/
          Max. process+thread count 887 (vs. ulimit of 10000)
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/22436/console
          Powered by Apache Yetus 0.8.0 http://yetus.apache.org

          This message was automatically generated.

          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.       trunk Compile Tests 0 mvndep 0m 11s Maven dependency ordering for branch +1 mvninstall 20m 9s trunk passed +1 compile 2m 47s trunk passed +1 checkstyle 1m 1s trunk passed +1 mvnsite 1m 16s trunk passed +1 shadedclient 14m 45s branch has no errors when building and testing our client artifacts. +1 findbugs 1m 57s trunk passed +1 javadoc 0m 49s trunk passed       Patch Compile Tests 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 1m 14s the patch passed +1 compile 2m 40s the patch passed +1 javac 2m 40s the patch passed +1 checkstyle 0m 54s the patch passed +1 mvnsite 1m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedclient 13m 49s patch has no errors when building and testing our client artifacts. +1 findbugs 2m 7s the patch passed +1 javadoc 0m 49s the patch passed       Other Tests -1 unit 108m 17s hadoop-yarn-server-resourcemanager in the patch failed. +1 unit 1m 48s hadoop-yarn-server-router in the patch passed. +1 asflicense 0m 25s The patch does not generate ASF License warnings. 175m 51s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f JIRA Issue YARN-8973 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12947135/YARN-8973.v3.patch Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle uname Linux e354aad4ccb6 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision trunk / 08d69d9 maven version: Apache Maven 3.3.9 Default Java 1.8.0_181 findbugs v3.1.0-RC1 unit https://builds.apache.org/job/PreCommit-YARN-Build/22436/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/22436/testReport/ Max. process+thread count 887 (vs. ulimit of 10000) modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server Console output https://builds.apache.org/job/PreCommit-YARN-Build/22436/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
          elgoiri Íñigo Goiri added a comment -

          Can we make the test in TestRouterWebServices a little more than just testing if the result is null?

          I'm not following the YARN unit tests much, is TestCapacityOverTimePolicy flaky?
          It failed in two of the runs.

          elgoiri Íñigo Goiri added a comment - Can we make the test in TestRouterWebServices a little more than just testing if the result is null? I'm not following the YARN unit tests much, is TestCapacityOverTimePolicy flaky? It failed in two of the runs.
          slfan1989 Shilun Fan added a comment -

          I will continue to follow up on this pr.

          slfan1989 Shilun Fan added a comment - I will continue to follow up on this pr.
          slfan1989 Shilun Fan added a comment -

          elgoiri Thank you very much for helping to review the code!

          Thanks everyone!

          slfan1989 Shilun Fan added a comment - elgoiri Thank you very much for helping to review the code! Thanks everyone!

          People

            slfan1989 Shilun Fan
            giovanni.fumarola Giovanni Matteo Fumarola
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 1h 20m
                1h 20m