Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-8842

Expose metrics for custom resource types in QueueMetrics

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.3.0, 3.2.2, 3.1.4, 2.10.1
    • None
    • None
    • Reviewed

    Description

      This is the 2nd dependent jira of YARN-8059.
      As updating the metrics is an independent step from handling preemption, this jira only deals with the queue metrics update of custom resources.
      The following metrics should be updated:

      • allocated resources
      • available resources
      • pending resources
      • reserved resources
      • aggregate seconds preempted

      Attachments

        1. YARN-8842.001.patch
          74 kB
          Szilard Nemeth
        2. YARN-8842.002.patch
          46 kB
          Szilard Nemeth
        3. YARN-8842.003.patch
          82 kB
          Szilard Nemeth
        4. YARN-8842.004.patch
          82 kB
          Szilard Nemeth
        5. YARN-8842.005.patch
          82 kB
          Szilard Nemeth
        6. YARN-8842.006.patch
          108 kB
          Szilard Nemeth
        7. YARN-8842.007.patch
          99 kB
          Szilard Nemeth
        8. YARN-8842.008.patch
          99 kB
          Szilard Nemeth
        9. YARN-8842.009.patch
          97 kB
          Szilard Nemeth
        10. YARN-8842.010.patch
          93 kB
          Szilard Nemeth
        11. YARN-8842.011.patch
          96 kB
          Szilard Nemeth
        12. YARN-8842.012.patch
          96 kB
          Szilard Nemeth
        13. YARN-8842-branch-2.001.patch
          102 kB
          Jonathan Hung
        14. YARN-8842-branch-2.002.patch
          102 kB
          Jonathan Hung
        15. YARN-8842-branch-2.003.patch
          102 kB
          Jonathan Hung

        Issue Links

          Activity

            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 0s Docker mode activated.
            -1 patch 0m 4s YARN-8842 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 4s YARN-8842 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Issue YARN-8842 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12942244/YARN-8842.001.patch Console output https://builds.apache.org/job/PreCommit-YARN-Build/22043/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
            wilfreds Wilfred Spiegelenburg added a comment - - edited

            Thank you for splitting this of from the pre-emption jira.

            I tried to apply the patch on top of YARN-8750 but it did not work...

            Beside that I went through the code as a diff and already have a couple of remarks:

            • You are always creating a new QueueMetricsForCustomResources. In the case that we just have two resources (mem and vcores) we will never use it and all metrics will stay zero. We do have the full overhead of making all the calls. From a performance perspective would it not be better to determine if we have custom resources configured in the constructor and set a flag to update the custom metrics or not? We can then make updating the custom resource metrics conditional which would keep the performance impact in most clusters to a minimum.
            • In updatePreemptedSecondsForCustomResources() you directly set the parent metrics but you do not take into account that there might be multiple levels above the queue you are working on. We need to call parent.updatePreemptedSecondsForCustomResources()
            • In getAllocatedResources() you're using BuilderUtils if you have no custom resources and a new resource object directly if you have. We should either use BuilderUtils in both cases or neither for consistency.
            • Again getAllocatedResources() when we have custom resource types (even if none are allocated) I would expect a metrics object back showing all the resource types not just memory and vcores. There should also be no need for isThereAnyAllocatedResource the Resource handles that itself.
            • In the test cases I could not find any multiple queue levels tests. root -> parent -> leaf. At least we should check that case with the root queue metrics to catch the issue in the updatePreemptedSecondsForCustomResources case from above.

            I'll dig further into the tests when we have a patch that applies.

            wilfreds Wilfred Spiegelenburg added a comment - - edited Thank you for splitting this of from the pre-emption jira. I tried to apply the patch on top of YARN-8750 but it did not work... Beside that I went through the code as a diff and already have a couple of remarks: You are always creating a new QueueMetricsForCustomResources. In the case that we just have two resources (mem and vcores) we will never use it and all metrics will stay zero. We do have the full overhead of making all the calls. From a performance perspective would it not be better to determine if we have custom resources configured in the constructor and set a flag to update the custom metrics or not? We can then make updating the custom resource metrics conditional which would keep the performance impact in most clusters to a minimum. In updatePreemptedSecondsForCustomResources() you directly set the parent metrics but you do not take into account that there might be multiple levels above the queue you are working on. We need to call parent.updatePreemptedSecondsForCustomResources() In getAllocatedResources() you're using BuilderUtils if you have no custom resources and a new resource object directly if you have. We should either use BuilderUtils in both cases or neither for consistency. Again getAllocatedResources() when we have custom resource types (even if none are allocated) I would expect a metrics object back showing all the resource types not just memory and vcores. There should also be no need for isThereAnyAllocatedResource the Resource handles that itself. In the test cases I could not find any multiple queue levels tests. root -> parent -> leaf. At least we should check that case with the root queue metrics to catch the issue in the updatePreemptedSecondsForCustomResources case from above. I'll dig further into the tests when we have a patch that applies.
            wilfreds Wilfred Spiegelenburg added a comment - - edited

            One further remark that showed up while reviewing YARN-8059: I don't think we should have the getMaxAllocationUtilization in the custom metrics and getMaxUtilizationOfCustomResources in the normal metrics. They pollute the classes with calculations that we should keep in the FairScheduler only.

            wilfreds Wilfred Spiegelenburg added a comment - - edited One further remark that showed up while reviewing YARN-8059 : I don't think we should have the getMaxAllocationUtilization in the custom metrics and getMaxUtilizationOfCustomResources in the normal metrics. They pollute the classes with calculations that we should keep in the FairScheduler only.
            snemeth Szilard Nemeth added a comment -

            Hi wilfreds!
            Thanks for the comments.
            1. Good idea to decide on the flag. However, it's not immediately straightforward for me whether we only need to treat memory / cores as default resources. See: https://github.com/apache/hadoop/blob/12a095a496dd59066d73a7a6c24129b5b6a9d650/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ResourceInformation.java#L56 , here we also have GPU / FPGA in the mandatory resources list.
            How would you exactly check in the constructor whether we have custom resources configured?
            I can think of the following: As we receive the configuration object in the constructor, we could check for how many keys are in there for YarnConfiguration.RESOURCE_TYPES. If it's 2 (4 in case we treat GPU / FPGA as standard resources), then we are not creating the QueueMetricsForCustomResources, otherwise we create it.
            Can you think of any better way to do this?
            2. It's a good catch, thanks!
            3. Good point, fixed.
            4. Refactored QueueMetricsTestcase in order to handle more than 2 queues, so that I can not only specify root and leaf queues, but multiple queues in a hierarchy.

            snemeth Szilard Nemeth added a comment - Hi wilfreds ! Thanks for the comments. 1. Good idea to decide on the flag. However, it's not immediately straightforward for me whether we only need to treat memory / cores as default resources. See: https://github.com/apache/hadoop/blob/12a095a496dd59066d73a7a6c24129b5b6a9d650/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ResourceInformation.java#L56 , here we also have GPU / FPGA in the mandatory resources list. How would you exactly check in the constructor whether we have custom resources configured? I can think of the following: As we receive the configuration object in the constructor, we could check for how many keys are in there for YarnConfiguration.RESOURCE_TYPES . If it's 2 (4 in case we treat GPU / FPGA as standard resources), then we are not creating the QueueMetricsForCustomResources , otherwise we create it. Can you think of any better way to do this? 2. It's a good catch, thanks! 3. Good point, fixed. 4. Refactored QueueMetricsTestcase in order to handle more than 2 queues, so that I can not only specify root and leaf queues, but multiple queues in a hierarchy.
            snemeth Szilard Nemeth added a comment -

            About your second comment: I think it does make sense to move that method with YARN-8059.

            snemeth Szilard Nemeth added a comment - About your second comment: I think it does make sense to move that method with YARN-8059 .
            haibochen Haibo Chen added a comment -

            Thanks snemeth for the patch.

            For a clean-code perspective, QueueMetricsForCustomResources can include memory and cpu metrics too. Though I am not sure how that's going to impact the compatibility (not sure if @Metric annotation is some sort of API) and performance that wilfreds is concerned with.

            We can also split QueueMetricsForCustomResources into three instances, Pending, Available and Allocated, so it is more readable.

            I'll take  another pass to look at the uni tests added later.

             

            haibochen Haibo Chen added a comment - Thanks snemeth for the patch. For a clean-code perspective, QueueMetricsForCustomResources can include memory and cpu metrics too. Though I am not sure how that's going to impact the compatibility (not sure if @Metric annotation is some sort of API) and performance that wilfreds is concerned with. We can also split QueueMetricsForCustomResources into three instances, Pending, Available and Allocated, so it is more readable. I'll take  another pass to look at the uni tests added later.  
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 27s Docker mode activated.
                  Prechecks
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
                  trunk Compile Tests
            +1 mvninstall 17m 37s trunk passed
            +1 compile 0m 42s trunk passed
            +1 checkstyle 0m 37s trunk passed
            +1 mvnsite 0m 47s trunk passed
            +1 shadedclient 12m 4s branch has no errors when building and testing our client artifacts.
            +1 findbugs 1m 10s trunk passed
            +1 javadoc 0m 29s trunk passed
                  Patch Compile Tests
            +1 mvninstall 0m 44s the patch passed
            +1 compile 0m 38s the patch passed
            +1 javac 0m 38s the patch passed
            -0 checkstyle 0m 33s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 42 new + 142 unchanged - 2 fixed = 184 total (was 144)
            +1 mvnsite 0m 43s the patch passed
            -1 whitespace 0m 0s The patch has 22 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
            +1 shadedclient 11m 38s patch has no errors when building and testing our client artifacts.
            +1 findbugs 1m 18s the patch passed
            +1 javadoc 0m 27s the patch passed
                  Other Tests
            -1 unit 89m 27s hadoop-yarn-server-resourcemanager in the patch failed.
            +1 asflicense 0m 26s The patch does not generate ASF License warnings.
            139m 36s



            Subsystem Report/Notes
            Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:4b8c2b1
            JIRA Issue YARN-8842
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12942640/YARN-8842.003.patch
            Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
            uname Linux 999dfc22c67c 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10 07:31:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/patchprocess/precommit/personality/provided.sh
            git revision trunk / cdf5d58
            maven version: Apache Maven 3.3.9
            Default Java 1.8.0_181
            findbugs v3.1.0-RC1
            checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/22076/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
            whitespace https://builds.apache.org/job/PreCommit-YARN-Build/22076/artifact/out/whitespace-eol.txt
            unit https://builds.apache.org/job/PreCommit-YARN-Build/22076/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
            Test Results https://builds.apache.org/job/PreCommit-YARN-Build/22076/testReport/
            Max. process+thread count 879 (vs. ulimit of 10000)
            modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
            Console output https://builds.apache.org/job/PreCommit-YARN-Build/22076/console
            Powered by Apache Yetus 0.8.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 27s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.       trunk Compile Tests +1 mvninstall 17m 37s trunk passed +1 compile 0m 42s trunk passed +1 checkstyle 0m 37s trunk passed +1 mvnsite 0m 47s trunk passed +1 shadedclient 12m 4s branch has no errors when building and testing our client artifacts. +1 findbugs 1m 10s trunk passed +1 javadoc 0m 29s trunk passed       Patch Compile Tests +1 mvninstall 0m 44s the patch passed +1 compile 0m 38s the patch passed +1 javac 0m 38s the patch passed -0 checkstyle 0m 33s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 42 new + 142 unchanged - 2 fixed = 184 total (was 144) +1 mvnsite 0m 43s the patch passed -1 whitespace 0m 0s The patch has 22 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 shadedclient 11m 38s patch has no errors when building and testing our client artifacts. +1 findbugs 1m 18s the patch passed +1 javadoc 0m 27s the patch passed       Other Tests -1 unit 89m 27s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 26s The patch does not generate ASF License warnings. 139m 36s Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:4b8c2b1 JIRA Issue YARN-8842 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12942640/YARN-8842.003.patch Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle uname Linux 999dfc22c67c 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10 07:31:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision trunk / cdf5d58 maven version: Apache Maven 3.3.9 Default Java 1.8.0_181 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/22076/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/22076/artifact/out/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/22076/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/22076/testReport/ Max. process+thread count 879 (vs. ulimit of 10000) modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/22076/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
            snemeth Szilard Nemeth added a comment -

            patch004 fixes newline and most of the checkstyle issues

            snemeth Szilard Nemeth added a comment - patch004 fixes newline and most of the checkstyle issues
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 0s Docker mode activated.
            -1 patch 0m 6s YARN-8842 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 6s YARN-8842 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Issue YARN-8842 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12942660/YARN-8842.004.patch Console output https://builds.apache.org/job/PreCommit-YARN-Build/22081/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 24s Docker mode activated.
                  Prechecks
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
                  trunk Compile Tests
            +1 mvninstall 17m 27s trunk passed
            +1 compile 0m 41s trunk passed
            +1 checkstyle 0m 35s trunk passed
            +1 mvnsite 0m 46s trunk passed
            +1 shadedclient 11m 11s branch has no errors when building and testing our client artifacts.
            +1 findbugs 1m 8s trunk passed
            +1 javadoc 0m 26s trunk passed
                  Patch Compile Tests
            +1 mvninstall 0m 45s the patch passed
            +1 compile 0m 38s the patch passed
            +1 javac 0m 38s the patch passed
            -0 checkstyle 0m 33s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 21 new + 142 unchanged - 2 fixed = 163 total (was 144)
            +1 mvnsite 0m 42s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 shadedclient 10m 59s patch has no errors when building and testing our client artifacts.
            +1 findbugs 1m 12s the patch passed
            +1 javadoc 0m 23s the patch passed
                  Other Tests
            -1 unit 89m 9s hadoop-yarn-server-resourcemanager in the patch failed.
            +1 asflicense 0m 25s The patch does not generate ASF License warnings.
            137m 9s



            Reason Tests
            Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueManagementDynamicEditPolicy



            Subsystem Report/Notes
            Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:4b8c2b1
            JIRA Issue YARN-8842
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12942681/YARN-8842.005.patch
            Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
            uname Linux 30b2dca12fea 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10 07:31:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/patchprocess/precommit/personality/provided.sh
            git revision trunk / cdf5d58
            maven version: Apache Maven 3.3.9
            Default Java 1.8.0_181
            findbugs v3.1.0-RC1
            checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/22084/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
            unit https://builds.apache.org/job/PreCommit-YARN-Build/22084/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
            Test Results https://builds.apache.org/job/PreCommit-YARN-Build/22084/testReport/
            Max. process+thread count 886 (vs. ulimit of 10000)
            modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
            Console output https://builds.apache.org/job/PreCommit-YARN-Build/22084/console
            Powered by Apache Yetus 0.8.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 24s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.       trunk Compile Tests +1 mvninstall 17m 27s trunk passed +1 compile 0m 41s trunk passed +1 checkstyle 0m 35s trunk passed +1 mvnsite 0m 46s trunk passed +1 shadedclient 11m 11s branch has no errors when building and testing our client artifacts. +1 findbugs 1m 8s trunk passed +1 javadoc 0m 26s trunk passed       Patch Compile Tests +1 mvninstall 0m 45s the patch passed +1 compile 0m 38s the patch passed +1 javac 0m 38s the patch passed -0 checkstyle 0m 33s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 21 new + 142 unchanged - 2 fixed = 163 total (was 144) +1 mvnsite 0m 42s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedclient 10m 59s patch has no errors when building and testing our client artifacts. +1 findbugs 1m 12s the patch passed +1 javadoc 0m 23s the patch passed       Other Tests -1 unit 89m 9s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 25s The patch does not generate ASF License warnings. 137m 9s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueManagementDynamicEditPolicy Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:4b8c2b1 JIRA Issue YARN-8842 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12942681/YARN-8842.005.patch Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle uname Linux 30b2dca12fea 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10 07:31:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision trunk / cdf5d58 maven version: Apache Maven 3.3.9 Default Java 1.8.0_181 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/22084/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/22084/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/22084/testReport/ Max. process+thread count 886 (vs. ulimit of 10000) modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/22084/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
            snemeth Szilard Nemeth added a comment -

            According to our offline discussion with haibochen, here are the changes incorporated with the latest patch (patch006).
            As I applied all the changes listed here as separate steps and ran the tests with every change, an unwanted code change is quite unlikely as the test were always passed.

            1. Unified resource decrease methods into one single method, so that:
            withContainersToDecrease, withVCoresToDecrease, withMemoryMBToDecrease, withCustomResToDecrease became withResourceToDecrease. This way, testcases are more concise and explicit.
            2. Same as above, but for the declaration of resources and not decrease of resources.
            2. Changed the QueueHierarchy class: instead of representing the hierarchy as a list of queue names, I created a Queue class that holds its name and a reference to its child queue. The validation code is also changed, an on-demand validation of queue names does happen when adding a child queue to the hierarchy.
            3. The common parts of initializing QueueMetricsTestcase was extracted to a method.
            4. Fixed test methods that only tested leaf queue metrics: now these methods get all the queue metrics and check the values on all queue levels. Example: testUpdatePreemptedSeconds previously only checked mqs.getLeafQueueSource, now it gets all the queue metrics sources and checks them.
            5. Created separate classes for every type of custom resource: preempted seconds, allocated, available, pending, reserved.
            Kept QueueMetricsForCustomResources class as a wrapper of all types, so that QueueMetrics should only refer to this class and it will delegate all alterations to the appropriate resource metric type objects.
            Having an abstract class as a parent for the metric type classes named QueueMetricsCustomResourcesAbstract, I could keep the computation methods in one single place.

            6. Added the performance optimization as wilfreds suggested: Custom resource metric update only happen if there is any custom resource defined.

            Please note that:
            1. The testcases in QueueMetricsForCustomResources would be more directly readable if the testXXX methods in QueueMetricsTestcase were moved to there. The reason why I don't do that is that QueueMetricsTestcase is intended to be a self-contained class that holds the data and the logic as well for all types of metrics testcases. Ideally, QueueMetricsTestcase could be used even from TestQueueMetrics, but I did not want to squeeze way too many changes into this patch.

            2. I think it's not a right decision to move all metrics for cores / memory to QueueMetricsForCustomResources, as the former are annotated Metrics, the latter are not real metrics, but values stored in a map for every resource.
            All in all, I like to idea to have all resource metrics in one class per type, but I would do that in a follow-up jira. One more thing to keep in mind is that AFAIK, the metric subsytem from Hadoop-common is not ready for using metrics from dynamic fields (e.g. Maps) so this will be quite tricky to implement.

            snemeth Szilard Nemeth added a comment - According to our offline discussion with haibochen , here are the changes incorporated with the latest patch (patch006). As I applied all the changes listed here as separate steps and ran the tests with every change, an unwanted code change is quite unlikely as the test were always passed. 1. Unified resource decrease methods into one single method, so that: withContainersToDecrease, withVCoresToDecrease, withMemoryMBToDecrease, withCustomResToDecrease became withResourceToDecrease. This way, testcases are more concise and explicit. 2. Same as above, but for the declaration of resources and not decrease of resources. 2. Changed the QueueHierarchy class: instead of representing the hierarchy as a list of queue names, I created a Queue class that holds its name and a reference to its child queue. The validation code is also changed, an on-demand validation of queue names does happen when adding a child queue to the hierarchy. 3. The common parts of initializing QueueMetricsTestcase was extracted to a method. 4. Fixed test methods that only tested leaf queue metrics: now these methods get all the queue metrics and check the values on all queue levels. Example: testUpdatePreemptedSeconds previously only checked mqs.getLeafQueueSource, now it gets all the queue metrics sources and checks them. 5. Created separate classes for every type of custom resource: preempted seconds, allocated, available, pending, reserved. Kept QueueMetricsForCustomResources class as a wrapper of all types, so that QueueMetrics should only refer to this class and it will delegate all alterations to the appropriate resource metric type objects. Having an abstract class as a parent for the metric type classes named QueueMetricsCustomResourcesAbstract, I could keep the computation methods in one single place. 6. Added the performance optimization as wilfreds suggested: Custom resource metric update only happen if there is any custom resource defined. Please note that: 1. The testcases in QueueMetricsForCustomResources would be more directly readable if the testXXX methods in QueueMetricsTestcase were moved to there. The reason why I don't do that is that QueueMetricsTestcase is intended to be a self-contained class that holds the data and the logic as well for all types of metrics testcases. Ideally, QueueMetricsTestcase could be used even from TestQueueMetrics, but I did not want to squeeze way too many changes into this patch. 2. I think it's not a right decision to move all metrics for cores / memory to QueueMetricsForCustomResources, as the former are annotated Metrics, the latter are not real metrics, but values stored in a map for every resource. All in all, I like to idea to have all resource metrics in one class per type, but I would do that in a follow-up jira. One more thing to keep in mind is that AFAIK, the metric subsytem from Hadoop-common is not ready for using metrics from dynamic fields (e.g. Maps) so this will be quite tricky to implement.
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 22s Docker mode activated.
                  Prechecks
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.
                  trunk Compile Tests
            0 mvndep 1m 13s Maven dependency ordering for branch
            +1 mvninstall 20m 22s trunk passed
            +1 compile 8m 57s trunk passed
            +1 checkstyle 1m 29s trunk passed
            +1 mvnsite 1m 44s trunk passed
            +1 shadedclient 15m 6s branch has no errors when building and testing our client artifacts.
            +1 findbugs 2m 47s trunk passed
            +1 javadoc 1m 36s trunk passed
                  Patch Compile Tests
            0 mvndep 0m 25s Maven dependency ordering for patch
            +1 mvninstall 1m 32s the patch passed
            +1 compile 12m 55s the patch passed
            +1 javac 12m 55s the patch passed
            -0 checkstyle 1m 41s hadoop-yarn-project/hadoop-yarn: The patch generated 42 new + 185 unchanged - 2 fixed = 227 total (was 187)
            +1 mvnsite 2m 3s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 shadedclient 13m 57s patch has no errors when building and testing our client artifacts.
            +1 findbugs 3m 35s the patch passed
            +1 javadoc 1m 50s the patch passed
                  Other Tests
            +1 unit 3m 48s hadoop-yarn-common in the patch passed.
            -1 unit 135m 55s hadoop-yarn-server-resourcemanager in the patch failed.
            +1 asflicense 1m 5s The patch does not generate ASF License warnings.
            231m 52s



            Reason Tests
            Failed junit tests hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens
              hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
              hadoop.yarn.server.resourcemanager.TestApplicationMasterServiceCapacity
              hadoop.yarn.server.resourcemanager.reservation.TestFairSchedulerPlanFollower
              hadoop.yarn.server.resourcemanager.resourcetracker.TestNMReconnect
              hadoop.yarn.server.resourcemanager.TestResourceTrackerService
              hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
              hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppAttempt
              hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler
              hadoop.yarn.server.resourcemanager.TestApplicationMasterServiceFair
              hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler
              hadoop.yarn.server.resourcemanager.reservation.TestReservationSystem
              hadoop.yarn.server.resourcemanager.scheduler.fair.TestApplicationMasterServiceWithFS
              hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart
              hadoop.yarn.server.resourcemanager.TestRM
              hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppStarvation
              hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerFairShare
              hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption
              hadoop.yarn.server.resourcemanager.scheduler.fair.TestAppRunnability
              hadoop.yarn.server.resourcemanager.scheduler.TestSchedulingWithAllocationRequestId
              hadoop.yarn.server.resourcemanager.rmapp.TestApplicationLifetimeMonitor
              hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSLeafQueue
              hadoop.yarn.server.resourcemanager.scheduler.fair.TestSchedulingUpdate
              hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter
              hadoop.yarn.server.resourcemanager.scheduler.fair.TestContinuousScheduling



            Subsystem Report/Notes
            Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:4b8c2b1
            JIRA Issue YARN-8842
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12943135/YARN-8842.006.patch
            Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
            uname Linux 76d63121c2cd 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/patchprocess/precommit/personality/provided.sh
            git revision trunk / 6a39739
            maven version: Apache Maven 3.3.9
            Default Java 1.8.0_181
            findbugs v3.1.0-RC1
            checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/22125/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
            unit https://builds.apache.org/job/PreCommit-YARN-Build/22125/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
            Test Results https://builds.apache.org/job/PreCommit-YARN-Build/22125/testReport/
            Max. process+thread count 970 (vs. ulimit of 10000)
            modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
            Console output https://builds.apache.org/job/PreCommit-YARN-Build/22125/console
            Powered by Apache Yetus 0.8.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.       trunk Compile Tests 0 mvndep 1m 13s Maven dependency ordering for branch +1 mvninstall 20m 22s trunk passed +1 compile 8m 57s trunk passed +1 checkstyle 1m 29s trunk passed +1 mvnsite 1m 44s trunk passed +1 shadedclient 15m 6s branch has no errors when building and testing our client artifacts. +1 findbugs 2m 47s trunk passed +1 javadoc 1m 36s trunk passed       Patch Compile Tests 0 mvndep 0m 25s Maven dependency ordering for patch +1 mvninstall 1m 32s the patch passed +1 compile 12m 55s the patch passed +1 javac 12m 55s the patch passed -0 checkstyle 1m 41s hadoop-yarn-project/hadoop-yarn: The patch generated 42 new + 185 unchanged - 2 fixed = 227 total (was 187) +1 mvnsite 2m 3s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedclient 13m 57s patch has no errors when building and testing our client artifacts. +1 findbugs 3m 35s the patch passed +1 javadoc 1m 50s the patch passed       Other Tests +1 unit 3m 48s hadoop-yarn-common in the patch passed. -1 unit 135m 55s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 1m 5s The patch does not generate ASF License warnings. 231m 52s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens   hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart   hadoop.yarn.server.resourcemanager.TestApplicationMasterServiceCapacity   hadoop.yarn.server.resourcemanager.reservation.TestFairSchedulerPlanFollower   hadoop.yarn.server.resourcemanager.resourcetracker.TestNMReconnect   hadoop.yarn.server.resourcemanager.TestResourceTrackerService   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppAttempt   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler   hadoop.yarn.server.resourcemanager.TestApplicationMasterServiceFair   hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler   hadoop.yarn.server.resourcemanager.reservation.TestReservationSystem   hadoop.yarn.server.resourcemanager.scheduler.fair.TestApplicationMasterServiceWithFS   hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart   hadoop.yarn.server.resourcemanager.TestRM   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppStarvation   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerFairShare   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption   hadoop.yarn.server.resourcemanager.scheduler.fair.TestAppRunnability   hadoop.yarn.server.resourcemanager.scheduler.TestSchedulingWithAllocationRequestId   hadoop.yarn.server.resourcemanager.rmapp.TestApplicationLifetimeMonitor   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSLeafQueue   hadoop.yarn.server.resourcemanager.scheduler.fair.TestSchedulingUpdate   hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter   hadoop.yarn.server.resourcemanager.scheduler.fair.TestContinuousScheduling Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:4b8c2b1 JIRA Issue YARN-8842 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12943135/YARN-8842.006.patch Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle uname Linux 76d63121c2cd 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision trunk / 6a39739 maven version: Apache Maven 3.3.9 Default Java 1.8.0_181 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/22125/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/22125/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/22125/testReport/ Max. process+thread count 970 (vs. ulimit of 10000) modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/22125/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
            snemeth Szilard Nemeth added a comment -

            patch008 fixes a missing null check

            snemeth Szilard Nemeth added a comment - patch008 fixes a missing null check
            wilfreds Wilfred Spiegelenburg added a comment - - edited

            Thank you snemeth. I am glad we gave this its own jira separate from the pre-emption changes. It has become quite large on its own.

            The patch overall looks good. I have a couple of remarks:

            • getMaxAllocationUtilization is new and not used outside of the patch, why would we add it if we're only going to leverage it in YARN-8059?
            • With that simplification we also do not need the tests for getMaxAllocationUtilization in the TestQueueMetricsForCustomResources
            • There was a possible NPE in getAllocatedResources until v7now fixed, combined with the fact that I do not understand the need for the isThereAnyAllocatedResource, the way the newInstance is created we can pass in an empty list or even a null. I would expect the method to look like this:
                  if (queueMetricsForCustomResources != null) {
                    return Resource.newInstance(allocatedMB.value(), allocatedVCores.value(),
                        queueMetricsForCustomResources.getAllocatedValues());
                  }
                  return Resource.newInstance(allocatedMB.value(), allocatedVCores.value());
              

              That does bring the QueueMetricsAllocatedCustomResources class back in line with all other custom resource metrics classes, just extend the abstract.

            • Please look at the checkstyle issues mentioned.

            A nit: on my system a number of indent inconsistencies showed up (mainly in TestQueueMetricsForCustomResources, QueueMetricsTestcase & TestQueueMetrics)

            wilfreds Wilfred Spiegelenburg added a comment - - edited Thank you snemeth . I am glad we gave this its own jira separate from the pre-emption changes. It has become quite large on its own. The patch overall looks good. I have a couple of remarks: getMaxAllocationUtilization is new and not used outside of the patch, why would we add it if we're only going to leverage it in YARN-8059 ? With that simplification we also do not need the tests for getMaxAllocationUtilization in the TestQueueMetricsForCustomResources There was a possible NPE in getAllocatedResources until v7now fixed, combined with the fact that I do not understand the need for the isThereAnyAllocatedResource , the way the newInstance is created we can pass in an empty list or even a null. I would expect the method to look like this: if (queueMetricsForCustomResources != null ) { return Resource.newInstance(allocatedMB.value(), allocatedVCores.value(), queueMetricsForCustomResources.getAllocatedValues()); } return Resource.newInstance(allocatedMB.value(), allocatedVCores.value()); That does bring the QueueMetricsAllocatedCustomResources class back in line with all other custom resource metrics classes, just extend the abstract. Please look at the checkstyle issues mentioned. A nit: on my system a number of indent inconsistencies showed up (mainly in TestQueueMetricsForCustomResources, QueueMetricsTestcase & TestQueueMetrics)
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 12s Docker mode activated.
                  Prechecks
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.
                  trunk Compile Tests
            0 mvndep 3m 22s Maven dependency ordering for branch
            +1 mvninstall 22m 0s trunk passed
            +1 compile 8m 34s trunk passed
            +1 checkstyle 1m 29s trunk passed
            +1 mvnsite 1m 46s trunk passed
            +1 shadedclient 15m 10s branch has no errors when building and testing our client artifacts.
            +1 findbugs 3m 1s trunk passed
            +1 javadoc 1m 33s trunk passed
                  Patch Compile Tests
            0 mvndep 0m 13s Maven dependency ordering for patch
            +1 mvninstall 1m 29s the patch passed
            +1 compile 7m 24s the patch passed
            +1 javac 7m 24s the patch passed
            -0 checkstyle 1m 26s hadoop-yarn-project/hadoop-yarn: The patch generated 37 new + 185 unchanged - 2 fixed = 222 total (was 187)
            +1 mvnsite 1m 48s the patch passed
            +1 whitespace 0m 1s The patch has no whitespace issues.
            +1 shadedclient 12m 17s patch has no errors when building and testing our client artifacts.
            +1 findbugs 2m 59s the patch passed
            +1 javadoc 1m 25s the patch passed
                  Other Tests
            +1 unit 3m 18s hadoop-yarn-common in the patch passed.
            -1 unit 122m 29s hadoop-yarn-server-resourcemanager in the patch failed.
            +1 asflicense 0m 41s The patch does not generate ASF License warnings.
            211m 35s



            Reason Tests
            Failed junit tests hadoop.yarn.server.resourcemanager.TestResourceTrackerService
              hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
              hadoop.yarn.server.resourcemanager.scheduler.fair.TestAppRunnability
              hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler
              hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppStarvation
              hadoop.yarn.server.resourcemanager.TestApplicationMasterServiceCapacity
              hadoop.yarn.server.resourcemanager.reservation.TestFairSchedulerPlanFollower
              hadoop.yarn.server.resourcemanager.rmapp.TestApplicationLifetimeMonitor
              hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSLeafQueue
              hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
              hadoop.yarn.server.resourcemanager.TestApplicationMasterServiceFair
              hadoop.yarn.server.resourcemanager.scheduler.fair.TestSchedulingUpdate
              hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens
              hadoop.yarn.server.resourcemanager.resourcetracker.TestNMReconnect
              hadoop.yarn.server.resourcemanager.reservation.TestReservationSystem
              hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter
              hadoop.yarn.server.resourcemanager.TestRM
              hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppAttempt
              hadoop.yarn.server.resourcemanager.scheduler.fair.TestContinuousScheduling
              hadoop.yarn.server.resourcemanager.scheduler.fair.TestApplicationMasterServiceWithFS
              hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart
              hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler
              hadoop.yarn.server.resourcemanager.scheduler.TestSchedulingWithAllocationRequestId
              hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption
              hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerFairShare



            Subsystem Report/Notes
            Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:4b8c2b1
            JIRA Issue YARN-8842
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12943154/YARN-8842.007.patch
            Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
            uname Linux bbd710f84f59 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27 10:45:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/patchprocess/precommit/personality/provided.sh
            git revision trunk / edce866
            maven version: Apache Maven 3.3.9
            Default Java 1.8.0_181
            findbugs v3.1.0-RC1
            checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/22127/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
            unit https://builds.apache.org/job/PreCommit-YARN-Build/22127/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
            Test Results https://builds.apache.org/job/PreCommit-YARN-Build/22127/testReport/
            Max. process+thread count 961 (vs. ulimit of 10000)
            modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
            Console output https://builds.apache.org/job/PreCommit-YARN-Build/22127/console
            Powered by Apache Yetus 0.8.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.       trunk Compile Tests 0 mvndep 3m 22s Maven dependency ordering for branch +1 mvninstall 22m 0s trunk passed +1 compile 8m 34s trunk passed +1 checkstyle 1m 29s trunk passed +1 mvnsite 1m 46s trunk passed +1 shadedclient 15m 10s branch has no errors when building and testing our client artifacts. +1 findbugs 3m 1s trunk passed +1 javadoc 1m 33s trunk passed       Patch Compile Tests 0 mvndep 0m 13s Maven dependency ordering for patch +1 mvninstall 1m 29s the patch passed +1 compile 7m 24s the patch passed +1 javac 7m 24s the patch passed -0 checkstyle 1m 26s hadoop-yarn-project/hadoop-yarn: The patch generated 37 new + 185 unchanged - 2 fixed = 222 total (was 187) +1 mvnsite 1m 48s the patch passed +1 whitespace 0m 1s The patch has no whitespace issues. +1 shadedclient 12m 17s patch has no errors when building and testing our client artifacts. +1 findbugs 2m 59s the patch passed +1 javadoc 1m 25s the patch passed       Other Tests +1 unit 3m 18s hadoop-yarn-common in the patch passed. -1 unit 122m 29s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 41s The patch does not generate ASF License warnings. 211m 35s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestResourceTrackerService   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification   hadoop.yarn.server.resourcemanager.scheduler.fair.TestAppRunnability   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppStarvation   hadoop.yarn.server.resourcemanager.TestApplicationMasterServiceCapacity   hadoop.yarn.server.resourcemanager.reservation.TestFairSchedulerPlanFollower   hadoop.yarn.server.resourcemanager.rmapp.TestApplicationLifetimeMonitor   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSLeafQueue   hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart   hadoop.yarn.server.resourcemanager.TestApplicationMasterServiceFair   hadoop.yarn.server.resourcemanager.scheduler.fair.TestSchedulingUpdate   hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens   hadoop.yarn.server.resourcemanager.resourcetracker.TestNMReconnect   hadoop.yarn.server.resourcemanager.reservation.TestReservationSystem   hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter   hadoop.yarn.server.resourcemanager.TestRM   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppAttempt   hadoop.yarn.server.resourcemanager.scheduler.fair.TestContinuousScheduling   hadoop.yarn.server.resourcemanager.scheduler.fair.TestApplicationMasterServiceWithFS   hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart   hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler   hadoop.yarn.server.resourcemanager.scheduler.TestSchedulingWithAllocationRequestId   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerFairShare Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:4b8c2b1 JIRA Issue YARN-8842 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12943154/YARN-8842.007.patch Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle uname Linux bbd710f84f59 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27 10:45:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision trunk / edce866 maven version: Apache Maven 3.3.9 Default Java 1.8.0_181 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/22127/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/22127/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/22127/testReport/ Max. process+thread count 961 (vs. ulimit of 10000) modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/22127/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 27s Docker mode activated.
                  Prechecks
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.
                  trunk Compile Tests
            0 mvndep 1m 22s Maven dependency ordering for branch
            +1 mvninstall 20m 16s trunk passed
            +1 compile 9m 0s trunk passed
            +1 checkstyle 1m 29s trunk passed
            +1 mvnsite 1m 43s trunk passed
            +1 shadedclient 14m 58s branch has no errors when building and testing our client artifacts.
            +1 findbugs 2m 48s trunk passed
            +1 javadoc 1m 26s trunk passed
                  Patch Compile Tests
            0 mvndep 0m 13s Maven dependency ordering for patch
            +1 mvninstall 1m 24s the patch passed
            +1 compile 8m 28s the patch passed
            +1 javac 8m 28s the patch passed
            -0 checkstyle 1m 25s hadoop-yarn-project/hadoop-yarn: The patch generated 37 new + 185 unchanged - 2 fixed = 222 total (was 187)
            +1 mvnsite 1m 39s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 shadedclient 12m 27s patch has no errors when building and testing our client artifacts.
            +1 findbugs 2m 59s the patch passed
            +1 javadoc 1m 23s the patch passed
                  Other Tests
            +1 unit 3m 16s hadoop-yarn-common in the patch passed.
            -1 unit 94m 34s hadoop-yarn-server-resourcemanager in the patch failed.
            +1 asflicense 1m 4s The patch does not generate ASF License warnings.
            181m 19s



            Subsystem Report/Notes
            Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:4b8c2b1
            JIRA Issue YARN-8842
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12943165/YARN-8842.008.patch
            Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
            uname Linux 36634744457f 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/patchprocess/precommit/personality/provided.sh
            git revision trunk / edce866
            maven version: Apache Maven 3.3.9
            Default Java 1.8.0_181
            findbugs v3.1.0-RC1
            checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/22129/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
            unit https://builds.apache.org/job/PreCommit-YARN-Build/22129/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
            Test Results https://builds.apache.org/job/PreCommit-YARN-Build/22129/testReport/
            Max. process+thread count 906 (vs. ulimit of 10000)
            modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
            Console output https://builds.apache.org/job/PreCommit-YARN-Build/22129/console
            Powered by Apache Yetus 0.8.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 27s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.       trunk Compile Tests 0 mvndep 1m 22s Maven dependency ordering for branch +1 mvninstall 20m 16s trunk passed +1 compile 9m 0s trunk passed +1 checkstyle 1m 29s trunk passed +1 mvnsite 1m 43s trunk passed +1 shadedclient 14m 58s branch has no errors when building and testing our client artifacts. +1 findbugs 2m 48s trunk passed +1 javadoc 1m 26s trunk passed       Patch Compile Tests 0 mvndep 0m 13s Maven dependency ordering for patch +1 mvninstall 1m 24s the patch passed +1 compile 8m 28s the patch passed +1 javac 8m 28s the patch passed -0 checkstyle 1m 25s hadoop-yarn-project/hadoop-yarn: The patch generated 37 new + 185 unchanged - 2 fixed = 222 total (was 187) +1 mvnsite 1m 39s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedclient 12m 27s patch has no errors when building and testing our client artifacts. +1 findbugs 2m 59s the patch passed +1 javadoc 1m 23s the patch passed       Other Tests +1 unit 3m 16s hadoop-yarn-common in the patch passed. -1 unit 94m 34s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 1m 4s The patch does not generate ASF License warnings. 181m 19s Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:4b8c2b1 JIRA Issue YARN-8842 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12943165/YARN-8842.008.patch Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle uname Linux 36634744457f 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision trunk / edce866 maven version: Apache Maven 3.3.9 Default Java 1.8.0_181 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/22129/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/22129/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/22129/testReport/ Max. process+thread count 906 (vs. ulimit of 10000) modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/22129/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
            wilfreds Wilfred Spiegelenburg added a comment - - edited

            The test failure is not related to this change as I see it happening in different builds. Based on the comments on YARN-8468 Weiwei thinks it is related to that change.

            Can you have a look at the comments I added, even after version 8 they are still relevant.

            wilfreds Wilfred Spiegelenburg added a comment - - edited The test failure is not related to this change as I see it happening in different builds. Based on the comments on YARN-8468 Weiwei thinks it is related to that change. Can you have a look at the comments I added, even after version 8 they are still relevant.
            snemeth Szilard Nemeth added a comment -

            Hi wilfreds!
            Thanks for coming back to this!
            I think I addressed all your comments, with this answer: https://issues.apache.org/jira/browse/YARN-8842?focusedCommentId=16639171&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16639171
            If you don't think so please list what is missing because I think all of them are addressed, except the getMaxAllocationUtilization method I haven't touched as I would prefer to do that with YARN-8059, which will be done immediately when this gets merged.
            Thanks!

            snemeth Szilard Nemeth added a comment - Hi wilfreds ! Thanks for coming back to this! I think I addressed all your comments, with this answer: https://issues.apache.org/jira/browse/YARN-8842?focusedCommentId=16639171&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16639171 If you don't think so please list what is missing because I think all of them are addressed, except the getMaxAllocationUtilization method I haven't touched as I would prefer to do that with YARN-8059 , which will be done immediately when this gets merged. Thanks!
            snemeth Szilard Nemeth added a comment -

            Based on our offline discussion with wilfreds, here are the changes included in patch009:

            • Removed getMaxAllocationUtilization from QueueMetricsForCustomResources and its related testcases and moved the code to YARN-8059 patch002
            • Removed the unnecessary check of queueMetricsForCustomResources.isThereAnyAllocatedResource in QueueMetrics#getAllocatedResources as Resource.newInstance already handles an empty-map.
            • Fixed all the checkstyle issues, except "'xxx' hides a field. [HiddenField]" kind of issues. This could only disappear if the parameter has a different name than the fields, which does not make sense for me.
            snemeth Szilard Nemeth added a comment - Based on our offline discussion with wilfreds , here are the changes included in patch009: Removed getMaxAllocationUtilization from QueueMetricsForCustomResources and its related testcases and moved the code to YARN-8059 patch002 Removed the unnecessary check of queueMetricsForCustomResources.isThereAnyAllocatedResource in QueueMetrics#getAllocatedResources as Resource.newInstance already handles an empty-map. Fixed all the checkstyle issues, except "'xxx' hides a field. [HiddenField] " kind of issues. This could only disappear if the parameter has a different name than the fields, which does not make sense for me.
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 15s Docker mode activated.
                  Prechecks
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.
                  trunk Compile Tests
            0 mvndep 1m 25s Maven dependency ordering for branch
            +1 mvninstall 20m 33s trunk passed
            +1 compile 9m 16s trunk passed
            +1 checkstyle 1m 30s trunk passed
            +1 mvnsite 1m 44s trunk passed
            +1 shadedclient 15m 3s branch has no errors when building and testing our client artifacts.
            +1 findbugs 2m 53s trunk passed
            +1 javadoc 1m 24s trunk passed
                  Patch Compile Tests
            0 mvndep 0m 13s Maven dependency ordering for patch
            +1 mvninstall 1m 36s the patch passed
            +1 compile 8m 17s the patch passed
            +1 javac 8m 17s the patch passed
            -0 checkstyle 1m 31s hadoop-yarn-project/hadoop-yarn: The patch generated 15 new + 185 unchanged - 2 fixed = 200 total (was 187)
            +1 mvnsite 1m 50s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 shadedclient 12m 27s patch has no errors when building and testing our client artifacts.
            +1 findbugs 3m 8s the patch passed
            +1 javadoc 1m 23s the patch passed
                  Other Tests
            +1 unit 3m 21s hadoop-yarn-common in the patch passed.
            -1 unit 82m 49s hadoop-yarn-server-resourcemanager in the patch failed.
            +1 asflicense 0m 40s The patch does not generate ASF License warnings.
            170m 11s



            Subsystem Report/Notes
            Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:4b8c2b1
            JIRA Issue YARN-8842
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12943495/YARN-8842.009.patch
            Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
            uname Linux afb0a586ff68 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/patchprocess/precommit/personality/provided.sh
            git revision trunk / 2addebb
            maven version: Apache Maven 3.3.9
            Default Java 1.8.0_181
            findbugs v3.1.0-RC1
            checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/22158/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
            unit https://builds.apache.org/job/PreCommit-YARN-Build/22158/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
            Test Results https://builds.apache.org/job/PreCommit-YARN-Build/22158/testReport/
            Max. process+thread count 866 (vs. ulimit of 10000)
            modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
            Console output https://builds.apache.org/job/PreCommit-YARN-Build/22158/console
            Powered by Apache Yetus 0.8.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.       trunk Compile Tests 0 mvndep 1m 25s Maven dependency ordering for branch +1 mvninstall 20m 33s trunk passed +1 compile 9m 16s trunk passed +1 checkstyle 1m 30s trunk passed +1 mvnsite 1m 44s trunk passed +1 shadedclient 15m 3s branch has no errors when building and testing our client artifacts. +1 findbugs 2m 53s trunk passed +1 javadoc 1m 24s trunk passed       Patch Compile Tests 0 mvndep 0m 13s Maven dependency ordering for patch +1 mvninstall 1m 36s the patch passed +1 compile 8m 17s the patch passed +1 javac 8m 17s the patch passed -0 checkstyle 1m 31s hadoop-yarn-project/hadoop-yarn: The patch generated 15 new + 185 unchanged - 2 fixed = 200 total (was 187) +1 mvnsite 1m 50s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedclient 12m 27s patch has no errors when building and testing our client artifacts. +1 findbugs 3m 8s the patch passed +1 javadoc 1m 23s the patch passed       Other Tests +1 unit 3m 21s hadoop-yarn-common in the patch passed. -1 unit 82m 49s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 40s The patch does not generate ASF License warnings. 170m 11s Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:4b8c2b1 JIRA Issue YARN-8842 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12943495/YARN-8842.009.patch Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle uname Linux afb0a586ff68 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision trunk / 2addebb maven version: Apache Maven 3.3.9 Default Java 1.8.0_181 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/22158/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/22158/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/22158/testReport/ Max. process+thread count 866 (vs. ulimit of 10000) modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/22158/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
            wilfreds Wilfred Spiegelenburg added a comment - - edited

            Thank you for the updated patch.
            I am OK with the patch as it is now v9, +1 non binding

            There are two minor checkstyle issues which could be fixed on checkin:

            • Class Queue should be declared as final. [FinalClass]
            • Name 'conf' must match pattern '^[A-Z][A-Z0-9](_[A-Z0-9]+)$'. [ConstantName]
            wilfreds Wilfred Spiegelenburg added a comment - - edited Thank you for the updated patch. I am OK with the patch as it is now v9, +1 non binding There are two minor checkstyle issues which could be fixed on checkin: Class Queue should be declared as final. [FinalClass] Name 'conf' must match pattern '^ [A-Z] [A-Z0-9] (_ [A-Z0-9] +) $'. [ConstantName]
            snemeth Szilard Nemeth added a comment -

            Hi wilfreds!
            Thanks for the review!

            snemeth Szilard Nemeth added a comment - Hi wilfreds ! Thanks for the review!
            snemeth Szilard Nemeth added a comment -

            I cleaned up the patch a little bit.
            Let's wait for jenkins with the results.

            snemeth Szilard Nemeth added a comment - I cleaned up the patch a little bit. Let's wait for jenkins with the results.
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 29s Docker mode activated.
                  Prechecks
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
                  trunk Compile Tests
            0 mvndep 1m 12s Maven dependency ordering for branch
            +1 mvninstall 19m 52s trunk passed
            +1 compile 9m 7s trunk passed
            +1 checkstyle 1m 30s trunk passed
            +1 mvnsite 1m 48s trunk passed
            +1 shadedclient 15m 32s branch has no errors when building and testing our client artifacts.
            +1 findbugs 2m 55s trunk passed
            +1 javadoc 1m 28s trunk passed
                  Patch Compile Tests
            0 mvndep 0m 12s Maven dependency ordering for patch
            +1 mvninstall 1m 30s the patch passed
            +1 compile 9m 4s the patch passed
            +1 javac 9m 4s the patch passed
            -0 checkstyle 1m 35s hadoop-yarn-project/hadoop-yarn: The patch generated 27 new + 142 unchanged - 2 fixed = 169 total (was 144)
            +1 mvnsite 1m 41s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 shadedclient 12m 56s patch has no errors when building and testing our client artifacts.
            +1 findbugs 3m 22s the patch passed
            +1 javadoc 1m 28s the patch passed
                  Other Tests
            +1 unit 3m 32s hadoop-yarn-common in the patch passed.
            -1 unit 87m 10s hadoop-yarn-server-resourcemanager in the patch failed.
            +1 asflicense 0m 45s The patch does not generate ASF License warnings.
            176m 4s



            Subsystem Report/Notes
            Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:4b8c2b1
            JIRA Issue YARN-8842
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12943734/YARN-8842.010.patch
            Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
            uname Linux f5c545dfcab0 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/patchprocess/precommit/personality/provided.sh
            git revision trunk / 28ca5c9
            maven version: Apache Maven 3.3.9
            Default Java 1.8.0_181
            findbugs v3.1.0-RC1
            checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/22173/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
            unit https://builds.apache.org/job/PreCommit-YARN-Build/22173/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
            Test Results https://builds.apache.org/job/PreCommit-YARN-Build/22173/testReport/
            Max. process+thread count 871 (vs. ulimit of 10000)
            modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
            Console output https://builds.apache.org/job/PreCommit-YARN-Build/22173/console
            Powered by Apache Yetus 0.8.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 29s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.       trunk Compile Tests 0 mvndep 1m 12s Maven dependency ordering for branch +1 mvninstall 19m 52s trunk passed +1 compile 9m 7s trunk passed +1 checkstyle 1m 30s trunk passed +1 mvnsite 1m 48s trunk passed +1 shadedclient 15m 32s branch has no errors when building and testing our client artifacts. +1 findbugs 2m 55s trunk passed +1 javadoc 1m 28s trunk passed       Patch Compile Tests 0 mvndep 0m 12s Maven dependency ordering for patch +1 mvninstall 1m 30s the patch passed +1 compile 9m 4s the patch passed +1 javac 9m 4s the patch passed -0 checkstyle 1m 35s hadoop-yarn-project/hadoop-yarn: The patch generated 27 new + 142 unchanged - 2 fixed = 169 total (was 144) +1 mvnsite 1m 41s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedclient 12m 56s patch has no errors when building and testing our client artifacts. +1 findbugs 3m 22s the patch passed +1 javadoc 1m 28s the patch passed       Other Tests +1 unit 3m 32s hadoop-yarn-common in the patch passed. -1 unit 87m 10s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 45s The patch does not generate ASF License warnings. 176m 4s Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:4b8c2b1 JIRA Issue YARN-8842 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12943734/YARN-8842.010.patch Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle uname Linux f5c545dfcab0 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision trunk / 28ca5c9 maven version: Apache Maven 3.3.9 Default Java 1.8.0_181 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/22173/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/22173/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/22173/testReport/ Max. process+thread count 871 (vs. ulimit of 10000) modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/22173/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
            snemeth Szilard Nemeth added a comment -

            Hi haibochen!
            Fixed some checkstyle issues with patch011.
            I realized there are a bunch of issues related to fields are not private and they don't have accessor methods in test classes. I think for these classes it's perfectly fine to access them as package-private fields.
            The other ones are for builder method parameters have the same name as the fields and they hide them: If I fixed this, it had no additional value and the param names could be too cryptic (abbrevations, for example).
            What do you think, do we have to fix any of the outstanding checkstyle issues?
            Thanks!

            snemeth Szilard Nemeth added a comment - Hi haibochen ! Fixed some checkstyle issues with patch011. I realized there are a bunch of issues related to fields are not private and they don't have accessor methods in test classes. I think for these classes it's perfectly fine to access them as package-private fields. The other ones are for builder method parameters have the same name as the fields and they hide them: If I fixed this, it had no additional value and the param names could be too cryptic (abbrevations, for example). What do you think, do we have to fix any of the outstanding checkstyle issues? Thanks!
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 26s Docker mode activated.
                  Prechecks
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
                  trunk Compile Tests
            0 mvndep 1m 8s Maven dependency ordering for branch
            +1 mvninstall 18m 11s trunk passed
            +1 compile 7m 58s trunk passed
            +1 checkstyle 1m 23s trunk passed
            +1 mvnsite 1m 46s trunk passed
            +1 shadedclient 14m 15s branch has no errors when building and testing our client artifacts.
            +1 findbugs 2m 49s trunk passed
            +1 javadoc 1m 29s trunk passed
                  Patch Compile Tests
            0 mvndep 0m 13s Maven dependency ordering for patch
            +1 mvninstall 1m 26s the patch passed
            +1 compile 7m 18s the patch passed
            +1 javac 7m 18s the patch passed
            -0 checkstyle 1m 19s hadoop-yarn-project/hadoop-yarn: The patch generated 21 new + 142 unchanged - 2 fixed = 163 total (was 144)
            +1 mvnsite 1m 40s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 shadedclient 11m 41s patch has no errors when building and testing our client artifacts.
            +1 findbugs 3m 3s the patch passed
            +1 javadoc 1m 25s the patch passed
                  Other Tests
            +1 unit 3m 37s hadoop-yarn-common in the patch passed.
            -1 unit 89m 22s hadoop-yarn-server-resourcemanager in the patch failed.
            +1 asflicense 0m 38s The patch does not generate ASF License warnings.
            170m 21s



            Subsystem Report/Notes
            Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:4b8c2b1
            JIRA Issue YARN-8842
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12943805/YARN-8842.011.patch
            Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
            uname Linux 53a570e520b0 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10 07:31:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/patchprocess/precommit/personality/provided.sh
            git revision trunk / 5209c75
            maven version: Apache Maven 3.3.9
            Default Java 1.8.0_181
            findbugs v3.1.0-RC1
            checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/22179/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
            unit https://builds.apache.org/job/PreCommit-YARN-Build/22179/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
            Test Results https://builds.apache.org/job/PreCommit-YARN-Build/22179/testReport/
            Max. process+thread count 937 (vs. ulimit of 10000)
            modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
            Console output https://builds.apache.org/job/PreCommit-YARN-Build/22179/console
            Powered by Apache Yetus 0.8.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 26s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.       trunk Compile Tests 0 mvndep 1m 8s Maven dependency ordering for branch +1 mvninstall 18m 11s trunk passed +1 compile 7m 58s trunk passed +1 checkstyle 1m 23s trunk passed +1 mvnsite 1m 46s trunk passed +1 shadedclient 14m 15s branch has no errors when building and testing our client artifacts. +1 findbugs 2m 49s trunk passed +1 javadoc 1m 29s trunk passed       Patch Compile Tests 0 mvndep 0m 13s Maven dependency ordering for patch +1 mvninstall 1m 26s the patch passed +1 compile 7m 18s the patch passed +1 javac 7m 18s the patch passed -0 checkstyle 1m 19s hadoop-yarn-project/hadoop-yarn: The patch generated 21 new + 142 unchanged - 2 fixed = 163 total (was 144) +1 mvnsite 1m 40s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedclient 11m 41s patch has no errors when building and testing our client artifacts. +1 findbugs 3m 3s the patch passed +1 javadoc 1m 25s the patch passed       Other Tests +1 unit 3m 37s hadoop-yarn-common in the patch passed. -1 unit 89m 22s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 38s The patch does not generate ASF License warnings. 170m 21s Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:4b8c2b1 JIRA Issue YARN-8842 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12943805/YARN-8842.011.patch Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle uname Linux 53a570e520b0 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10 07:31:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision trunk / 5209c75 maven version: Apache Maven 3.3.9 Default Java 1.8.0_181 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/22179/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/22179/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/22179/testReport/ Max. process+thread count 937 (vs. ulimit of 10000) modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/22179/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
            haibochen Haibo Chen added a comment -

            Thanks snemeth for updating the patch!  The patch looks much better to me. In addition to the checkstyle issues (I think we can ignore the ones like "HiddenField" and "must be private and have accessor methods"), I have one other minor comment: In QueueInfo, all fields can be marked as final I think. Let's also rename the methods following the pattern assert*() to check*().  +1 otherwise.

            haibochen Haibo Chen added a comment - Thanks snemeth for updating the patch!  The patch looks much better to me. In addition to the checkstyle issues (I think we can ignore the ones like "HiddenField" and "must be private and have accessor methods"), I have one other minor comment: In QueueInfo, all fields can be marked as final I think. Let's also rename the methods following the pattern assert* () to check *().  +1 otherwise.
            snemeth Szilard Nemeth added a comment -

            Hi haibochen!
            I applied the changes you suggested, please check the latest patch!
            Thanks!

            snemeth Szilard Nemeth added a comment - Hi haibochen ! I applied the changes you suggested, please check the latest patch! Thanks!
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 20s Docker mode activated.
                  Prechecks
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
                  trunk Compile Tests
            0 mvndep 1m 19s Maven dependency ordering for branch
            +1 mvninstall 18m 2s trunk passed
            +1 compile 8m 8s trunk passed
            +1 checkstyle 1m 12s trunk passed
            +1 mvnsite 1m 31s trunk passed
            +1 shadedclient 13m 12s branch has no errors when building and testing our client artifacts.
            +1 findbugs 2m 40s trunk passed
            +1 javadoc 1m 14s trunk passed
                  Patch Compile Tests
            0 mvndep 0m 14s Maven dependency ordering for patch
            +1 mvninstall 1m 23s the patch passed
            +1 compile 7m 33s the patch passed
            +1 javac 7m 33s the patch passed
            -0 checkstyle 1m 13s hadoop-yarn-project/hadoop-yarn: The patch generated 21 new + 142 unchanged - 2 fixed = 163 total (was 144)
            +1 mvnsite 1m 33s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 shadedclient 11m 37s patch has no errors when building and testing our client artifacts.
            +1 findbugs 3m 1s the patch passed
            +1 javadoc 1m 25s the patch passed
                  Other Tests
            +1 unit 3m 35s hadoop-yarn-common in the patch passed.
            -1 unit 88m 26s hadoop-yarn-server-resourcemanager in the patch failed.
            +1 asflicense 0m 41s The patch does not generate ASF License warnings.
            167m 54s



            Subsystem Report/Notes
            Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:4b8c2b1
            JIRA Issue YARN-8842
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12944044/YARN-8842.012.patch
            Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
            uname Linux c93c3e1523fa 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10 07:31:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/patchprocess/precommit/personality/provided.sh
            git revision trunk / 2614078
            maven version: Apache Maven 3.3.9
            Default Java 1.8.0_181
            findbugs v3.1.0-RC1
            checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/22191/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
            unit https://builds.apache.org/job/PreCommit-YARN-Build/22191/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
            Test Results https://builds.apache.org/job/PreCommit-YARN-Build/22191/testReport/
            Max. process+thread count 935 (vs. ulimit of 10000)
            modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
            Console output https://builds.apache.org/job/PreCommit-YARN-Build/22191/console
            Powered by Apache Yetus 0.8.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.       trunk Compile Tests 0 mvndep 1m 19s Maven dependency ordering for branch +1 mvninstall 18m 2s trunk passed +1 compile 8m 8s trunk passed +1 checkstyle 1m 12s trunk passed +1 mvnsite 1m 31s trunk passed +1 shadedclient 13m 12s branch has no errors when building and testing our client artifacts. +1 findbugs 2m 40s trunk passed +1 javadoc 1m 14s trunk passed       Patch Compile Tests 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 23s the patch passed +1 compile 7m 33s the patch passed +1 javac 7m 33s the patch passed -0 checkstyle 1m 13s hadoop-yarn-project/hadoop-yarn: The patch generated 21 new + 142 unchanged - 2 fixed = 163 total (was 144) +1 mvnsite 1m 33s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedclient 11m 37s patch has no errors when building and testing our client artifacts. +1 findbugs 3m 1s the patch passed +1 javadoc 1m 25s the patch passed       Other Tests +1 unit 3m 35s hadoop-yarn-common in the patch passed. -1 unit 88m 26s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 41s The patch does not generate ASF License warnings. 167m 54s Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:4b8c2b1 JIRA Issue YARN-8842 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12944044/YARN-8842.012.patch Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle uname Linux c93c3e1523fa 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10 07:31:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision trunk / 2614078 maven version: Apache Maven 3.3.9 Default Java 1.8.0_181 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/22191/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/22191/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/22191/testReport/ Max. process+thread count 935 (vs. ulimit of 10000) modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/22191/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 22s Docker mode activated.
                  Prechecks
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
                  trunk Compile Tests
            0 mvndep 3m 32s Maven dependency ordering for branch
            +1 mvninstall 19m 45s trunk passed
            +1 compile 9m 29s trunk passed
            +1 checkstyle 1m 21s trunk passed
            +1 mvnsite 1m 43s trunk passed
            +1 shadedclient 14m 9s branch has no errors when building and testing our client artifacts.
            +1 findbugs 2m 48s trunk passed
            +1 javadoc 1m 22s trunk passed
                  Patch Compile Tests
            0 mvndep 0m 12s Maven dependency ordering for patch
            +1 mvninstall 1m 22s the patch passed
            +1 compile 7m 26s the patch passed
            +1 javac 7m 26s the patch passed
            -0 checkstyle 1m 17s hadoop-yarn-project/hadoop-yarn: The patch generated 21 new + 142 unchanged - 2 fixed = 163 total (was 144)
            +1 mvnsite 1m 34s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 shadedclient 14m 46s patch has no errors when building and testing our client artifacts.
            +1 findbugs 3m 1s the patch passed
            +1 javadoc 1m 25s the patch passed
                  Other Tests
            +1 unit 3m 37s hadoop-yarn-common in the patch passed.
            -1 unit 90m 13s hadoop-yarn-server-resourcemanager in the patch failed.
            +1 asflicense 0m 40s The patch does not generate ASF License warnings.
            179m 14s



            Subsystem Report/Notes
            Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:4b8c2b1
            JIRA Issue YARN-8842
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12944044/YARN-8842.012.patch
            Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
            uname Linux 428b30eba13e 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10 07:31:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/patchprocess/precommit/personality/provided.sh
            git revision trunk / 2614078
            maven version: Apache Maven 3.3.9
            Default Java 1.8.0_181
            findbugs v3.1.0-RC1
            checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/22193/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
            unit https://builds.apache.org/job/PreCommit-YARN-Build/22193/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
            Test Results https://builds.apache.org/job/PreCommit-YARN-Build/22193/testReport/
            Max. process+thread count 960 (vs. ulimit of 10000)
            modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
            Console output https://builds.apache.org/job/PreCommit-YARN-Build/22193/console
            Powered by Apache Yetus 0.8.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.       trunk Compile Tests 0 mvndep 3m 32s Maven dependency ordering for branch +1 mvninstall 19m 45s trunk passed +1 compile 9m 29s trunk passed +1 checkstyle 1m 21s trunk passed +1 mvnsite 1m 43s trunk passed +1 shadedclient 14m 9s branch has no errors when building and testing our client artifacts. +1 findbugs 2m 48s trunk passed +1 javadoc 1m 22s trunk passed       Patch Compile Tests 0 mvndep 0m 12s Maven dependency ordering for patch +1 mvninstall 1m 22s the patch passed +1 compile 7m 26s the patch passed +1 javac 7m 26s the patch passed -0 checkstyle 1m 17s hadoop-yarn-project/hadoop-yarn: The patch generated 21 new + 142 unchanged - 2 fixed = 163 total (was 144) +1 mvnsite 1m 34s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedclient 14m 46s patch has no errors when building and testing our client artifacts. +1 findbugs 3m 1s the patch passed +1 javadoc 1m 25s the patch passed       Other Tests +1 unit 3m 37s hadoop-yarn-common in the patch passed. -1 unit 90m 13s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 40s The patch does not generate ASF License warnings. 179m 14s Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:4b8c2b1 JIRA Issue YARN-8842 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12944044/YARN-8842.012.patch Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle uname Linux 428b30eba13e 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10 07:31:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision trunk / 2614078 maven version: Apache Maven 3.3.9 Default Java 1.8.0_181 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/22193/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/22193/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/22193/testReport/ Max. process+thread count 960 (vs. ulimit of 10000) modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/22193/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
            haibochen Haibo Chen added a comment -

            +1 on the latest patch. I'll fix the one minor indentation checkstyle issues at the time of my commit.

            haibochen Haibo Chen added a comment - +1 on the latest patch. I'll fix the one minor indentation checkstyle issues at the time of my commit.
            haibochen Haibo Chen added a comment -

            Thanks snemeth for the contribution and wilfreds for additional reviews.

            haibochen Haibo Chen added a comment - Thanks snemeth for the contribution and wilfreds for additional reviews.
            hudson Hudson added a comment -

            SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15232 (See https://builds.apache.org/job/Hadoop-trunk-Commit/15232/)
            YARN-8842. Expose metrics for custom resource types in QueueMetrics. (haibochen: rev 84e22a6af46db2859d7d2caf192861cae9b6a1a8)

            • (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetricsForCustomResources.java
            • (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueInfo.java
            • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/ResourceMetricsChecker.java
            • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
            • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/resourcetypes/ResourceTypesTestHelper.java
            • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetrics.java
            • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestQueueMetrics.java
            • (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetricsTestData.java
            • (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestQueueMetricsForCustomResources.java
            hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15232 (See https://builds.apache.org/job/Hadoop-trunk-Commit/15232/ ) YARN-8842 . Expose metrics for custom resource types in QueueMetrics. (haibochen: rev 84e22a6af46db2859d7d2caf192861cae9b6a1a8) (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetricsForCustomResources.java (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueInfo.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/ResourceMetricsChecker.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/resourcetypes/ResourceTypesTestHelper.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetrics.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestQueueMetrics.java (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetricsTestData.java (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestQueueMetricsForCustomResources.java
            snemeth Szilard Nemeth added a comment -

            Thanks haibochen for the commit and for fixing the checkstyle indentation issue!

            snemeth Szilard Nemeth added a comment - Thanks haibochen for the commit and for fixing the checkstyle indentation issue!
            epayne Eric Payne added a comment -

            I'd like to backport this to branch-2 if there are no objections.

            epayne Eric Payne added a comment - I'd like to backport this to branch-2 if there are no objections.
            epayne Eric Payne added a comment -

            I backported to branch-3.2 and 3.1, but putting it into branch-2 seems to involve a tangled web of prerequisites.

            epayne Eric Payne added a comment - I backported to branch-3.2 and 3.1, but putting it into branch-2 seems to involve a tangled web of prerequisites.
            epayne Eric Payne added a comment -

            Hi snemeth. I would like this functionality to be backported to branch-2. Lambda functions are used extensively in the trunk patch, which is a problem since we must still support Java 1.7 in branch-2.

            Would you like to work on a branch-2 patch, or should I give it a try.

            epayne Eric Payne added a comment - Hi snemeth . I would like this functionality to be backported to branch-2. Lambda functions are used extensively in the trunk patch, which is a problem since we must still support Java 1.7 in branch-2. Would you like to work on a branch-2 patch, or should I give it a try.
            jhung Jonathan Hung added a comment - - edited

            epayne regarding the tangled web of dependencies...I've ported YARN-7739, YARN-7541, YARN-8202 to branch-2. Now YARN-8842 seems to apply (mostly) cleanly.

            The 1.7 support is not gonna be pretty though... There's a lot of 1.8 specific syntax in TestQueueMetricsForCustomResources and QueueInfo which passes methods into refactored functions. I can try to tackle this, let me know if you've already started work on this.

            jhung Jonathan Hung added a comment - - edited epayne  regarding the tangled web of dependencies...I've ported YARN-7739 , YARN-7541 , YARN-8202 to branch-2. Now YARN-8842 seems to apply (mostly) cleanly. The 1.7 support is not gonna be pretty though... There's a lot of 1.8 specific syntax in TestQueueMetricsForCustomResources and QueueInfo which passes methods into refactored functions. I can try to tackle this, let me know if you've already started work on this.
            jhung Jonathan Hung added a comment -

            I've attached YARN-8842-branch-2.001.patch. Main differences are :

            • Replacing expected resource maps with manually constructed ones, e.g: computeExpectedCustomResourceValues(testData.customResourceValues, (k, v) -> v * containers becomes
            +    Map<String, Long> expected = new HashMap<>();
            +    for (Map.Entry<String, Long> entry : testData.customResourceValues.entrySet()) {
            +      expected.put(entry.getKey(), entry.getValue() * containers);
            +    }
            • Duplicating all of the assertAllMetrics, assertQueueMetricsOnly, assertAllQueueMetrics, checkAllQueueMetrics four times, each one looks like assertAllXMetrics, assertQueueXMetricsOnly, assertAllXQueueMetrics, checkAllXQueueMetrics where X = {Pending, Allocated, Preempted, Reserved}

            epayne/snemeth could you help review? Thanks!

            jhung Jonathan Hung added a comment - I've attached YARN-8842-branch-2.001.patch . Main differences are : Replacing expected resource maps with manually constructed ones, e.g: computeExpectedCustomResourceValues(testData.customResourceValues, (k, v) -> v * containers becomes +    Map<String, Long> expected = new HashMap<>(); +    for (Map.Entry<String, Long> entry : testData.customResourceValues.entrySet()) { +      expected.put(entry.getKey(), entry.getValue() * containers); +    } Duplicating all of the assertAllMetrics , assertQueueMetricsOnly , assertAllQueueMetrics , checkAllQueueMetrics four times, each one looks like  assertAllXMetrics , assertQueueXMetricsOnly , assertAllXQueueMetrics , checkAllXQueueMetrics where X = {Pending, Allocated, Preempted, Reserved} epayne / snemeth  could you help review? Thanks!
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 31m 16s Docker mode activated.
                  Prechecks
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
                  branch-2 Compile Tests
            0 mvndep 0m 34s Maven dependency ordering for branch
            +1 mvninstall 14m 50s branch-2 passed
            +1 compile 9m 44s branch-2 passed with JDK v1.7.0_95
            +1 compile 8m 33s branch-2 passed with JDK v1.8.0_222
            +1 checkstyle 1m 8s branch-2 passed
            +1 mvnsite 1m 58s branch-2 passed
            +1 findbugs 3m 26s branch-2 passed
            +1 javadoc 1m 59s branch-2 passed with JDK v1.7.0_95
            +1 javadoc 1m 33s branch-2 passed with JDK v1.8.0_222
                  Patch Compile Tests
            0 mvndep 0m 14s Maven dependency ordering for patch
            +1 mvninstall 1m 14s the patch passed
            -1 compile 2m 16s hadoop-yarn in the patch failed with JDK v1.7.0_95.
            -1 javac 2m 16s hadoop-yarn in the patch failed with JDK v1.7.0_95.
            +1 compile 5m 43s the patch passed with JDK v1.8.0_222
            +1 javac 5m 43s the patch passed
            -0 checkstyle 0m 52s hadoop-yarn-project/hadoop-yarn: The patch generated 32 new + 139 unchanged - 2 fixed = 171 total (was 141)
            +1 mvnsite 1m 23s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 findbugs 2m 39s the patch passed
            -1 javadoc 0m 34s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 4 unchanged - 0 fixed = 5 total (was 4)
            +1 javadoc 1m 5s the patch passed with JDK v1.8.0_222
                  Other Tests
            -1 unit 3m 4s hadoop-yarn-common in the patch failed.
            +1 unit 65m 3s hadoop-yarn-server-resourcemanager in the patch passed.
            +1 asflicense 0m 38s The patch does not generate ASF License warnings.
            163m 53s



            Reason Tests
            Failed junit tests hadoop.yarn.api.TestPBImplRecords



            Subsystem Report/Notes
            Docker Client=19.03.5 Server=19.03.5 Image:yetus/hadoop:f555aa740b5
            JIRA Issue YARN-8842
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12985905/YARN-8842-branch-2.001.patch
            Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
            uname Linux 5e429cbae251 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/patchprocess/precommit/personality/provided.sh
            git revision branch-2 / 235d6c7
            maven version: Apache Maven 3.3.9
            Default Java 1.8.0_222
            Multi-JDK versions /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 /usr/lib/jvm/java-8-openjdk-amd64:1.8.0_222
            findbugs v3.0.0
            compile https://builds.apache.org/job/PreCommit-YARN-Build/25174/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.7.0_95.txt
            javac https://builds.apache.org/job/PreCommit-YARN-Build/25174/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.7.0_95.txt
            checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/25174/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
            javadoc https://builds.apache.org/job/PreCommit-YARN-Build/25174/artifact/out/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
            unit https://builds.apache.org/job/PreCommit-YARN-Build/25174/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
            Test Results https://builds.apache.org/job/PreCommit-YARN-Build/25174/testReport/
            Max. process+thread count 838 (vs. ulimit of 5500)
            modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
            Console output https://builds.apache.org/job/PreCommit-YARN-Build/25174/console
            Powered by Apache Yetus 0.8.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 31m 16s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.       branch-2 Compile Tests 0 mvndep 0m 34s Maven dependency ordering for branch +1 mvninstall 14m 50s branch-2 passed +1 compile 9m 44s branch-2 passed with JDK v1.7.0_95 +1 compile 8m 33s branch-2 passed with JDK v1.8.0_222 +1 checkstyle 1m 8s branch-2 passed +1 mvnsite 1m 58s branch-2 passed +1 findbugs 3m 26s branch-2 passed +1 javadoc 1m 59s branch-2 passed with JDK v1.7.0_95 +1 javadoc 1m 33s branch-2 passed with JDK v1.8.0_222       Patch Compile Tests 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 14s the patch passed -1 compile 2m 16s hadoop-yarn in the patch failed with JDK v1.7.0_95. -1 javac 2m 16s hadoop-yarn in the patch failed with JDK v1.7.0_95. +1 compile 5m 43s the patch passed with JDK v1.8.0_222 +1 javac 5m 43s the patch passed -0 checkstyle 0m 52s hadoop-yarn-project/hadoop-yarn: The patch generated 32 new + 139 unchanged - 2 fixed = 171 total (was 141) +1 mvnsite 1m 23s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 39s the patch passed -1 javadoc 0m 34s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 4 unchanged - 0 fixed = 5 total (was 4) +1 javadoc 1m 5s the patch passed with JDK v1.8.0_222       Other Tests -1 unit 3m 4s hadoop-yarn-common in the patch failed. +1 unit 65m 3s hadoop-yarn-server-resourcemanager in the patch passed. +1 asflicense 0m 38s The patch does not generate ASF License warnings. 163m 53s Reason Tests Failed junit tests hadoop.yarn.api.TestPBImplRecords Subsystem Report/Notes Docker Client=19.03.5 Server=19.03.5 Image:yetus/hadoop:f555aa740b5 JIRA Issue YARN-8842 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12985905/YARN-8842-branch-2.001.patch Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle uname Linux 5e429cbae251 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision branch-2 / 235d6c7 maven version: Apache Maven 3.3.9 Default Java 1.8.0_222 Multi-JDK versions /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 /usr/lib/jvm/java-8-openjdk-amd64:1.8.0_222 findbugs v3.0.0 compile https://builds.apache.org/job/PreCommit-YARN-Build/25174/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.7.0_95.txt javac https://builds.apache.org/job/PreCommit-YARN-Build/25174/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.7.0_95.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/25174/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/25174/artifact/out/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/25174/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/25174/testReport/ Max. process+thread count 838 (vs. ulimit of 5500) modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/25174/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
            jhung Jonathan Hung added a comment -

            Unit test failure is related to YARN-7589, I'll port that after this Jira goes into branch-2.

            YARN-8842-branch-2.002.patch fixes the javac/javadoc/compile error.

            jhung Jonathan Hung added a comment - Unit test failure is related to  YARN-7589 , I'll port that after this Jira goes into branch-2. YARN-8842-branch-2.002.patch fixes the javac/javadoc/compile error.
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 42s Docker mode activated.
                  Prechecks
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
                  branch-2 Compile Tests
            0 mvndep 0m 11s Maven dependency ordering for branch
            +1 mvninstall 8m 40s branch-2 passed
            +1 compile 6m 48s branch-2 passed with JDK v1.7.0_95
            +1 compile 5m 52s branch-2 passed with JDK v1.8.0_222
            +1 checkstyle 0m 52s branch-2 passed
            +1 mvnsite 1m 27s branch-2 passed
            +1 findbugs 2m 27s branch-2 passed
            +1 javadoc 1m 20s branch-2 passed with JDK v1.7.0_95
            +1 javadoc 1m 9s branch-2 passed with JDK v1.8.0_222
                  Patch Compile Tests
            0 mvndep 0m 11s Maven dependency ordering for patch
            +1 mvninstall 1m 8s the patch passed
            -1 compile 2m 18s hadoop-yarn in the patch failed with JDK v1.7.0_95.
            -1 javac 2m 18s hadoop-yarn in the patch failed with JDK v1.7.0_95.
            +1 compile 5m 29s the patch passed with JDK v1.8.0_222
            +1 javac 5m 29s the patch passed
            -0 checkstyle 0m 51s hadoop-yarn-project/hadoop-yarn: The patch generated 31 new + 139 unchanged - 2 fixed = 170 total (was 141)
            +1 mvnsite 1m 27s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 findbugs 3m 0s the patch passed
            +1 javadoc 1m 32s the patch passed with JDK v1.7.0_95
            +1 javadoc 1m 21s the patch passed with JDK v1.8.0_222
                  Other Tests
            -1 unit 3m 17s hadoop-yarn-common in the patch failed.
            +1 unit 64m 37s hadoop-yarn-server-resourcemanager in the patch passed.
            +1 asflicense 0m 35s The patch does not generate ASF License warnings.
            118m 44s



            Reason Tests
            Failed junit tests hadoop.yarn.api.TestPBImplRecords



            Subsystem Report/Notes
            Docker Client=19.03.5 Server=19.03.5 Image:yetus/hadoop:f555aa740b5
            JIRA Issue YARN-8842
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12985919/YARN-8842-branch-2.002.patch
            Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
            uname Linux e665b96675e9 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/patchprocess/precommit/personality/provided.sh
            git revision branch-2 / 81060b3
            maven version: Apache Maven 3.3.9
            Default Java 1.8.0_222
            Multi-JDK versions /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 /usr/lib/jvm/java-8-openjdk-amd64:1.8.0_222
            findbugs v3.0.0
            compile https://builds.apache.org/job/PreCommit-YARN-Build/25176/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.7.0_95.txt
            javac https://builds.apache.org/job/PreCommit-YARN-Build/25176/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.7.0_95.txt
            checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/25176/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
            unit https://builds.apache.org/job/PreCommit-YARN-Build/25176/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
            Test Results https://builds.apache.org/job/PreCommit-YARN-Build/25176/testReport/
            Max. process+thread count 838 (vs. ulimit of 5500)
            modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
            Console output https://builds.apache.org/job/PreCommit-YARN-Build/25176/console
            Powered by Apache Yetus 0.8.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 42s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.       branch-2 Compile Tests 0 mvndep 0m 11s Maven dependency ordering for branch +1 mvninstall 8m 40s branch-2 passed +1 compile 6m 48s branch-2 passed with JDK v1.7.0_95 +1 compile 5m 52s branch-2 passed with JDK v1.8.0_222 +1 checkstyle 0m 52s branch-2 passed +1 mvnsite 1m 27s branch-2 passed +1 findbugs 2m 27s branch-2 passed +1 javadoc 1m 20s branch-2 passed with JDK v1.7.0_95 +1 javadoc 1m 9s branch-2 passed with JDK v1.8.0_222       Patch Compile Tests 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 1m 8s the patch passed -1 compile 2m 18s hadoop-yarn in the patch failed with JDK v1.7.0_95. -1 javac 2m 18s hadoop-yarn in the patch failed with JDK v1.7.0_95. +1 compile 5m 29s the patch passed with JDK v1.8.0_222 +1 javac 5m 29s the patch passed -0 checkstyle 0m 51s hadoop-yarn-project/hadoop-yarn: The patch generated 31 new + 139 unchanged - 2 fixed = 170 total (was 141) +1 mvnsite 1m 27s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 0s the patch passed +1 javadoc 1m 32s the patch passed with JDK v1.7.0_95 +1 javadoc 1m 21s the patch passed with JDK v1.8.0_222       Other Tests -1 unit 3m 17s hadoop-yarn-common in the patch failed. +1 unit 64m 37s hadoop-yarn-server-resourcemanager in the patch passed. +1 asflicense 0m 35s The patch does not generate ASF License warnings. 118m 44s Reason Tests Failed junit tests hadoop.yarn.api.TestPBImplRecords Subsystem Report/Notes Docker Client=19.03.5 Server=19.03.5 Image:yetus/hadoop:f555aa740b5 JIRA Issue YARN-8842 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12985919/YARN-8842-branch-2.002.patch Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle uname Linux e665b96675e9 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision branch-2 / 81060b3 maven version: Apache Maven 3.3.9 Default Java 1.8.0_222 Multi-JDK versions /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 /usr/lib/jvm/java-8-openjdk-amd64:1.8.0_222 findbugs v3.0.0 compile https://builds.apache.org/job/PreCommit-YARN-Build/25176/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.7.0_95.txt javac https://builds.apache.org/job/PreCommit-YARN-Build/25176/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.7.0_95.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/25176/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/25176/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/25176/testReport/ Max. process+thread count 838 (vs. ulimit of 5500) modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/25176/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
            epayne Eric Payne added a comment -

            Wow, thanks a lot, jhung!

            I'll hopefully be able to get to the review early next week.

            epayne Eric Payne added a comment - Wow, thanks a lot, jhung ! I'll hopefully be able to get to the review early next week.
            jhung Jonathan Hung added a comment -

            Thanks epayne! Uploading YARN-8842-branch-2.003.patch to fix another javac  issue ...

            jhung Jonathan Hung added a comment - Thanks epayne ! Uploading YARN-8842-branch-2.003.patch to fix another javac  issue ...
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 56s Docker mode activated.
                  Prechecks
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
                  branch-2 Compile Tests
            0 mvndep 0m 16s Maven dependency ordering for branch
            +1 mvninstall 9m 39s branch-2 passed
            +1 compile 6m 41s branch-2 passed with JDK v1.7.0_95
            +1 compile 5m 47s branch-2 passed with JDK v1.8.0_222
            +1 checkstyle 0m 52s branch-2 passed
            +1 mvnsite 1m 33s branch-2 passed
            +1 findbugs 2m 55s branch-2 passed
            +1 javadoc 1m 27s branch-2 passed with JDK v1.7.0_95
            +1 javadoc 1m 19s branch-2 passed with JDK v1.8.0_222
                  Patch Compile Tests
            0 mvndep 0m 14s Maven dependency ordering for patch
            +1 mvninstall 1m 20s the patch passed
            +1 compile 6m 43s the patch passed with JDK v1.7.0_95
            +1 javac 6m 43s the patch passed
            +1 compile 5m 44s the patch passed with JDK v1.8.0_222
            +1 javac 5m 44s the patch passed
            -0 checkstyle 0m 51s hadoop-yarn-project/hadoop-yarn: The patch generated 30 new + 139 unchanged - 2 fixed = 169 total (was 141)
            +1 mvnsite 1m 25s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 findbugs 3m 1s the patch passed
            +1 javadoc 1m 17s the patch passed with JDK v1.7.0_95
            +1 javadoc 1m 4s the patch passed with JDK v1.8.0_222
                  Other Tests
            -1 unit 3m 2s hadoop-yarn-common in the patch failed.
            -1 unit 64m 18s hadoop-yarn-server-resourcemanager in the patch failed.
            +1 asflicense 0m 35s The patch does not generate ASF License warnings.
            124m 12s



            Reason Tests
            Failed junit tests hadoop.yarn.api.TestPBImplRecords
              hadoop.yarn.server.resourcemanager.TestApplicationMasterService
              hadoop.yarn.server.resourcemanager.scheduler.TestQueueMetricsForCustomResources
              hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerUtils



            Subsystem Report/Notes
            Docker Client=19.03.5 Server=19.03.5 Image:yetus/hadoop:f555aa740b5
            JIRA Issue YARN-8842
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12985981/YARN-8842-branch-2.003.patch
            Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
            uname Linux 5d382bd7580b 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/patchprocess/precommit/personality/provided.sh
            git revision branch-2 / 6973f78
            maven version: Apache Maven 3.3.9
            Default Java 1.8.0_222
            Multi-JDK versions /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 /usr/lib/jvm/java-8-openjdk-amd64:1.8.0_222
            findbugs v3.0.0
            checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/25182/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
            unit https://builds.apache.org/job/PreCommit-YARN-Build/25182/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
            unit https://builds.apache.org/job/PreCommit-YARN-Build/25182/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
            Test Results https://builds.apache.org/job/PreCommit-YARN-Build/25182/testReport/
            Max. process+thread count 860 (vs. ulimit of 5500)
            modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
            Console output https://builds.apache.org/job/PreCommit-YARN-Build/25182/console
            Powered by Apache Yetus 0.8.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 56s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.       branch-2 Compile Tests 0 mvndep 0m 16s Maven dependency ordering for branch +1 mvninstall 9m 39s branch-2 passed +1 compile 6m 41s branch-2 passed with JDK v1.7.0_95 +1 compile 5m 47s branch-2 passed with JDK v1.8.0_222 +1 checkstyle 0m 52s branch-2 passed +1 mvnsite 1m 33s branch-2 passed +1 findbugs 2m 55s branch-2 passed +1 javadoc 1m 27s branch-2 passed with JDK v1.7.0_95 +1 javadoc 1m 19s branch-2 passed with JDK v1.8.0_222       Patch Compile Tests 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 20s the patch passed +1 compile 6m 43s the patch passed with JDK v1.7.0_95 +1 javac 6m 43s the patch passed +1 compile 5m 44s the patch passed with JDK v1.8.0_222 +1 javac 5m 44s the patch passed -0 checkstyle 0m 51s hadoop-yarn-project/hadoop-yarn: The patch generated 30 new + 139 unchanged - 2 fixed = 169 total (was 141) +1 mvnsite 1m 25s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 1s the patch passed +1 javadoc 1m 17s the patch passed with JDK v1.7.0_95 +1 javadoc 1m 4s the patch passed with JDK v1.8.0_222       Other Tests -1 unit 3m 2s hadoop-yarn-common in the patch failed. -1 unit 64m 18s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 35s The patch does not generate ASF License warnings. 124m 12s Reason Tests Failed junit tests hadoop.yarn.api.TestPBImplRecords   hadoop.yarn.server.resourcemanager.TestApplicationMasterService   hadoop.yarn.server.resourcemanager.scheduler.TestQueueMetricsForCustomResources   hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerUtils Subsystem Report/Notes Docker Client=19.03.5 Server=19.03.5 Image:yetus/hadoop:f555aa740b5 JIRA Issue YARN-8842 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12985981/YARN-8842-branch-2.003.patch Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle uname Linux 5d382bd7580b 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision branch-2 / 6973f78 maven version: Apache Maven 3.3.9 Default Java 1.8.0_222 Multi-JDK versions /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 /usr/lib/jvm/java-8-openjdk-amd64:1.8.0_222 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/25182/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/25182/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/25182/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/25182/testReport/ Max. process+thread count 860 (vs. ulimit of 5500) modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/25182/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
            jhung Jonathan Hung added a comment -

            I think the additional hadoop-yarn-server-resourcemanager test failures were caused by YARN-7411 (which has since been reverted from branch-2: https://issues.apache.org/jira/browse/YARN-7411?focusedCommentId=16975442&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16975442)

            Rerunning jenkins.

            jhung Jonathan Hung added a comment - I think the additional hadoop-yarn-server-resourcemanager test failures were caused by YARN-7411 (which has since been reverted from branch-2:  https://issues.apache.org/jira/browse/YARN-7411?focusedCommentId=16975442&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16975442 ) Rerunning jenkins.
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 44s Docker mode activated.
                  Prechecks
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
                  branch-2 Compile Tests
            0 mvndep 0m 24s Maven dependency ordering for branch
            +1 mvninstall 12m 40s branch-2 passed
            +1 compile 6m 50s branch-2 passed with JDK v1.7.0_95
            +1 compile 5m 47s branch-2 passed with JDK v1.8.0_222
            +1 checkstyle 0m 54s branch-2 passed
            +1 mvnsite 1m 30s branch-2 passed
            +1 findbugs 2m 29s branch-2 passed
            +1 javadoc 1m 22s branch-2 passed with JDK v1.7.0_95
            +1 javadoc 1m 8s branch-2 passed with JDK v1.8.0_222
                  Patch Compile Tests
            0 mvndep 0m 11s Maven dependency ordering for patch
            +1 mvninstall 1m 6s the patch passed
            +1 compile 6m 10s the patch passed with JDK v1.7.0_95
            +1 javac 6m 10s the patch passed
            +1 compile 5m 41s the patch passed with JDK v1.8.0_222
            +1 javac 5m 41s the patch passed
            -0 checkstyle 0m 51s hadoop-yarn-project/hadoop-yarn: The patch generated 30 new + 139 unchanged - 2 fixed = 169 total (was 141)
            +1 mvnsite 1m 23s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 findbugs 2m 37s the patch passed
            +1 javadoc 1m 15s the patch passed with JDK v1.7.0_95
            +1 javadoc 1m 3s the patch passed with JDK v1.8.0_222
                  Other Tests
            -1 unit 3m 2s hadoop-yarn-common in the patch failed.
            +1 unit 63m 59s hadoop-yarn-server-resourcemanager in the patch passed.
            +1 asflicense 0m 35s The patch does not generate ASF License warnings.
            124m 49s



            Reason Tests
            Failed junit tests hadoop.yarn.api.TestPBImplRecords



            Subsystem Report/Notes
            Docker Client=19.03.5 Server=19.03.5 Image:yetus/hadoop:f555aa740b5
            JIRA Issue YARN-8842
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12985981/YARN-8842-branch-2.003.patch
            Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
            uname Linux e3e7e4a9a353 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/patchprocess/precommit/personality/provided.sh
            git revision branch-2 / a74411e
            maven version: Apache Maven 3.3.9
            Default Java 1.8.0_222
            Multi-JDK versions /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 /usr/lib/jvm/java-8-openjdk-amd64:1.8.0_222
            findbugs v3.0.0
            checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/25184/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
            unit https://builds.apache.org/job/PreCommit-YARN-Build/25184/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
            Test Results https://builds.apache.org/job/PreCommit-YARN-Build/25184/testReport/
            Max. process+thread count 893 (vs. ulimit of 5500)
            modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
            Console output https://builds.apache.org/job/PreCommit-YARN-Build/25184/console
            Powered by Apache Yetus 0.8.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 44s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.       branch-2 Compile Tests 0 mvndep 0m 24s Maven dependency ordering for branch +1 mvninstall 12m 40s branch-2 passed +1 compile 6m 50s branch-2 passed with JDK v1.7.0_95 +1 compile 5m 47s branch-2 passed with JDK v1.8.0_222 +1 checkstyle 0m 54s branch-2 passed +1 mvnsite 1m 30s branch-2 passed +1 findbugs 2m 29s branch-2 passed +1 javadoc 1m 22s branch-2 passed with JDK v1.7.0_95 +1 javadoc 1m 8s branch-2 passed with JDK v1.8.0_222       Patch Compile Tests 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 1m 6s the patch passed +1 compile 6m 10s the patch passed with JDK v1.7.0_95 +1 javac 6m 10s the patch passed +1 compile 5m 41s the patch passed with JDK v1.8.0_222 +1 javac 5m 41s the patch passed -0 checkstyle 0m 51s hadoop-yarn-project/hadoop-yarn: The patch generated 30 new + 139 unchanged - 2 fixed = 169 total (was 141) +1 mvnsite 1m 23s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 37s the patch passed +1 javadoc 1m 15s the patch passed with JDK v1.7.0_95 +1 javadoc 1m 3s the patch passed with JDK v1.8.0_222       Other Tests -1 unit 3m 2s hadoop-yarn-common in the patch failed. +1 unit 63m 59s hadoop-yarn-server-resourcemanager in the patch passed. +1 asflicense 0m 35s The patch does not generate ASF License warnings. 124m 49s Reason Tests Failed junit tests hadoop.yarn.api.TestPBImplRecords Subsystem Report/Notes Docker Client=19.03.5 Server=19.03.5 Image:yetus/hadoop:f555aa740b5 JIRA Issue YARN-8842 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12985981/YARN-8842-branch-2.003.patch Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle uname Linux e3e7e4a9a353 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision branch-2 / a74411e maven version: Apache Maven 3.3.9 Default Java 1.8.0_222 Multi-JDK versions /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 /usr/lib/jvm/java-8-openjdk-amd64:1.8.0_222 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/25184/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/25184/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/25184/testReport/ Max. process+thread count 893 (vs. ulimit of 5500) modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/25184/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
            epayne Eric Payne added a comment -

            Thanks for the patch, jhung.

            +1

            I will run some more tests and commit to branch-2 and branch-2.10 if no objections.

            epayne Eric Payne added a comment - Thanks for the patch, jhung . +1 I will run some more tests and commit to branch-2 and branch-2.10 if no objections.
            epayne Eric Payne added a comment -

            Thanks jhung and snemeth for all of your work on this JIRA. I committed YARN-8842-branch-2.003.patch to branch-2.

            I would also like to port this back to branch-2.10. jhung, if you don't have any objections, I will cherry-pick YARN-7739, YARN-7541, YARN-8202, which are prerequi8sites.

            epayne Eric Payne added a comment - Thanks jhung and snemeth for all of your work on this JIRA. I committed YARN-8842 -branch-2.003.patch to branch-2. I would also like to port this back to branch-2.10. jhung , if you don't have any objections, I will cherry-pick YARN-7739 , YARN-7541 , YARN-8202 , which are prerequi8sites.
            jhung Jonathan Hung added a comment -

            Thanks epayne, no objections from me!

            jhung Jonathan Hung added a comment - Thanks epayne , no objections from me!
            epayne Eric Payne added a comment -

            I committed this and the aforementioned JIRAs to branch-2.10 as well.

            Thanks for all your efforts, jhung

            epayne Eric Payne added a comment - I committed this and the aforementioned JIRAs to branch-2.10 as well. Thanks for all your efforts, jhung
            jhung Jonathan Hung added a comment -

            Removing 2.11.0 fix version after branch-2 -> branch-2.10 rename

            jhung Jonathan Hung added a comment - Removing 2.11.0 fix version after branch-2 -> branch-2.10 rename

            People

              snemeth Szilard Nemeth
              snemeth Szilard Nemeth
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: