Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.3.0
    • None
    • Reviewed

    Description

      Currently we just generated c-s.xml, but we didn't validate that. To make sure the c-s.xml is correct after conversion, it's better to initialize the CS scheduler using configs.

      Also, in the test, we should try to leverage MockRM to validate generated configs as much as we could.

      Attachments

        1. YARN-10167-005.patch
          35 kB
          Peter Bacsko
        2. YARN-10167-004.patch
          35 kB
          Peter Bacsko
        3. YARN-10167-003.patch
          32 kB
          Peter Bacsko
        4. YARN-10167-002.patch
          14 kB
          Peter Bacsko
        5. YARN-10167-001.patch
          13 kB
          Peter Bacsko

        Activity

          kmarton Kinga Marton added a comment -

          There is a validation API implemented in YARN-10022. We can use that API to validate the generated configuration.

          kmarton Kinga Marton added a comment - There is a validation API implemented in  YARN-10022 . We can use that API to validate the generated configuration.
          leftnoteasy Wangda Tan added a comment -

          Kinga: we may not be able to use that since we can not assume there’s a running cluster with CS enabled, we have to do this validation from the CLI itself.

          leftnoteasy Wangda Tan added a comment - Kinga: we may not be able to use that since we can not assume there’s a running cluster with CS enabled, we have to do this validation from the CLI itself.
          sunilg Sunil G added a comment -

          Trying to understand deeper level here

          • Submit a new c-s.xml from CLI  for validation (may be cli may look like: yarn queues validate -f c-s.xml)
          • CLI uploads the c-s.xml as a config object to RM
          • In RM, we will be having a method in ClientRMService called validateCSConfig
          • In this method, we will be able to get a new CS object itself, and inject the incoming conf object. 
            • 1. If CS can be inited, validation seems fine (similar to what kmarton did earlier)
            • 2. Can we start CS also ?? I dont think so. In this case, we may need MockRM instead of CS object. But question here is that, what all configs again get validated more when we call start. If this delta is minimum, I would like to pull that out and call explicitly. cc kmarton and prabhujoseph can add more color here.
            • If both 1 & 2 can be done, then we can say that the submitted config is good or not.
          • Propagate success or failure back to CLI

          leftnoteasy kmarton cud u pls share ur thoughts whether this is what intended?

          cc. pbacsko snemeth prabhujoseph

          sunilg Sunil G added a comment - Trying to understand deeper level here Submit a new c-s.xml from CLI  for validation (may be cli may look like: yarn queues validate -f c-s.xml ) CLI uploads the c-s.xml as a config object to RM In RM, we will be having a method in ClientRMService called validateCSConfig In this method, we will be able to get a new CS object itself, and inject the incoming conf object.  1. If CS can be inited, validation seems fine (similar to what kmarton  did earlier) 2. Can we start CS also ?? I dont think so. In this case, we may need MockRM instead of CS object. But question here is that, what all configs again get validated more when we call start. If this delta is minimum, I would like to pull that out and call explicitly. cc kmarton  and prabhujoseph  can add more color here. If both 1 & 2 can be done, then we can say that the submitted config is good or not. Propagate success or failure back to CLI leftnoteasy kmarton  cud u pls share ur thoughts whether this is what intended? cc. pbacsko snemeth prabhujoseph
          pbacsko Peter Bacsko added a comment -

          I think the command can be part of yarn fs2cs tool, with a switch, like --validate-cs-config or whatever.

          pbacsko Peter Bacsko added a comment - I think the command can be part of yarn fs2cs tool, with a switch, like --validate-cs-config or whatever.
          pbacsko Peter Bacsko added a comment - - edited

          sunilg it's way too complicated. IMO we don't need to contact the RM for validation. As leftnoteasy said, the cluster might be down.

          Note that the converter itself already starts an FS instance inside to parse and load the allocation file. We can do the same thing with CS. Just load the converted config along with the delta yarn-site.xml (which essentially means that we merge the original site + the delta) and let's see if it can start. If not, we might have a problem and the configuration needs adjustments. Otherwise it's good (at least from a syntactic perspective).

          pbacsko Peter Bacsko added a comment - - edited sunilg it's way too complicated. IMO we don't need to contact the RM for validation. As leftnoteasy said, the cluster might be down. Note that the converter itself already starts an FS instance inside to parse and load the allocation file. We can do the same thing with CS. Just load the converted config along with the delta yarn-site.xml (which essentially means that we merge the original site + the delta) and let's see if it can start. If not, we might have a problem and the configuration needs adjustments. Otherwise it's good (at least from a syntactic perspective).
          sunilg Sunil G added a comment -

          pbacsko thanks for pointing out the Cluster Down scenario. I missed that.

          If the fs2cs tool can bringup a CS instance, then its much better. FYI, kmarton has done similar effort for YARN validate mutation api call. So similar code will help here.

          Thoughts?

          sunilg Sunil G added a comment - pbacsko  thanks for pointing out the Cluster Down scenario. I missed that. If the fs2cs tool can bringup a CS instance, then its much better. FYI, kmarton  has done similar effort for YARN validate mutation api call. So similar code will help here. Thoughts?
          wangda Wangda Tan added a comment -

          pbacsko, agree with this: 

          Note that the converter itself already starts an FS instance inside to parse and load the allocation file. We can do the same thing with CS. Just load the converted config along with the delta yarn-site.xml (which essentially means that we merge the original site + the delta) and let's see if it can start.

          We can check if MiniYARNCluster can help or not. I'm not sure if we can directly initialize CS or not, since it has other module dependencies. 

          wangda Wangda Tan added a comment - pbacsko , agree with this:  Note that the converter itself already starts an FS instance inside to parse and load the allocation file. We can do the same thing with CS. Just load the converted config along with the delta  yarn-site.xml  (which essentially means that we merge the original site + the delta) and let's see if it can start. We can check if MiniYARNCluster can help or not. I'm not sure if we can directly initialize CS or not, since it has other module dependencies. 
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 40s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
                trunk Compile Tests
          +1 mvninstall 20m 41s trunk passed
          +1 compile 0m 43s trunk passed
          +1 checkstyle 0m 33s trunk passed
          +1 mvnsite 0m 44s trunk passed
          +1 shadedclient 15m 33s branch has no errors when building and testing our client artifacts.
          +1 findbugs 1m 34s trunk passed
          +1 javadoc 0m 30s trunk passed
                Patch Compile Tests
          +1 mvninstall 0m 42s the patch passed
          +1 compile 0m 37s the patch passed
          +1 javac 0m 37s the patch passed
          +1 checkstyle 0m 28s the patch passed
          +1 mvnsite 0m 42s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 shadedclient 14m 22s patch has no errors when building and testing our client artifacts.
          +1 findbugs 1m 36s the patch passed
          +1 javadoc 0m 27s the patch passed
                Other Tests
          -1 unit 89m 30s hadoop-yarn-server-resourcemanager in the patch passed.
          -1 asflicense 0m 29s The patch generated 1 ASF License warnings.
          149m 47s



          Reason Tests
          Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.fair.converter.TestFSQueueConverter
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs



          Subsystem Report/Notes
          Docker Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3
          JIRA Issue YARN-10167
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12995266/YARN-10167-001.patch
          Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
          uname Linux 507da6d03669 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/patchprocess/precommit/personality/provided.sh
          git revision trunk / 1a636da
          maven version: Apache Maven 3.3.9
          Default Java 1.8.0_242
          findbugs v3.1.0-RC1
          unit https://builds.apache.org/job/PreCommit-YARN-Build/25611/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/25611/testReport/
          asflicense https://builds.apache.org/job/PreCommit-YARN-Build/25611/artifact/out/patch-asflicense-problems.txt
          Max. process+thread count 826 (vs. ulimit of 5500)
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/25611/console
          Powered by Apache Yetus 0.8.0 http://yetus.apache.org

          This message was automatically generated.

          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 40s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.       trunk Compile Tests +1 mvninstall 20m 41s trunk passed +1 compile 0m 43s trunk passed +1 checkstyle 0m 33s trunk passed +1 mvnsite 0m 44s trunk passed +1 shadedclient 15m 33s branch has no errors when building and testing our client artifacts. +1 findbugs 1m 34s trunk passed +1 javadoc 0m 30s trunk passed       Patch Compile Tests +1 mvninstall 0m 42s the patch passed +1 compile 0m 37s the patch passed +1 javac 0m 37s the patch passed +1 checkstyle 0m 28s the patch passed +1 mvnsite 0m 42s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedclient 14m 22s patch has no errors when building and testing our client artifacts. +1 findbugs 1m 36s the patch passed +1 javadoc 0m 27s the patch passed       Other Tests -1 unit 89m 30s hadoop-yarn-server-resourcemanager in the patch passed. -1 asflicense 0m 29s The patch generated 1 ASF License warnings. 149m 47s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.fair.converter.TestFSQueueConverter   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs Subsystem Report/Notes Docker Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 JIRA Issue YARN-10167 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12995266/YARN-10167-001.patch Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle uname Linux 507da6d03669 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision trunk / 1a636da maven version: Apache Maven 3.3.9 Default Java 1.8.0_242 findbugs v3.1.0-RC1 unit https://builds.apache.org/job/PreCommit-YARN-Build/25611/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/25611/testReport/ asflicense https://builds.apache.org/job/PreCommit-YARN-Build/25611/artifact/out/patch-asflicense-problems.txt Max. process+thread count 826 (vs. ulimit of 5500) modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/25611/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 48s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
                trunk Compile Tests
          +1 mvninstall 20m 30s trunk passed
          +1 compile 0m 43s trunk passed
          +1 checkstyle 0m 33s trunk passed
          +1 mvnsite 0m 46s trunk passed
          +1 shadedclient 15m 29s branch has no errors when building and testing our client artifacts.
          +1 findbugs 1m 33s trunk passed
          +1 javadoc 0m 30s trunk passed
                Patch Compile Tests
          +1 mvninstall 0m 42s the patch passed
          +1 compile 0m 37s the patch passed
          +1 javac 0m 37s the patch passed
          +1 checkstyle 0m 29s the patch passed
          +1 mvnsite 0m 41s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 shadedclient 14m 18s patch has no errors when building and testing our client artifacts.
          +1 findbugs 1m 37s the patch passed
          +1 javadoc 0m 26s the patch passed
                Other Tests
          -1 unit 45m 39s hadoop-yarn-server-resourcemanager in the patch passed.
          0 asflicense 0m 32s ASF License check generated no output?
          105m 43s



          Reason Tests
          Failed junit tests hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart



          Subsystem Report/Notes
          Docker Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3
          JIRA Issue YARN-10167
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12995313/YARN-10167-002.patch
          Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
          uname Linux 32d186bade95 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/patchprocess/precommit/personality/provided.sh
          git revision trunk / edc2e9d
          maven version: Apache Maven 3.3.9
          Default Java 1.8.0_242
          findbugs v3.1.0-RC1
          unit https://builds.apache.org/job/PreCommit-YARN-Build/25615/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/25615/testReport/
          Max. process+thread count 839 (vs. ulimit of 5500)
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/25615/console
          Powered by Apache Yetus 0.8.0 http://yetus.apache.org

          This message was automatically generated.

          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 48s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.       trunk Compile Tests +1 mvninstall 20m 30s trunk passed +1 compile 0m 43s trunk passed +1 checkstyle 0m 33s trunk passed +1 mvnsite 0m 46s trunk passed +1 shadedclient 15m 29s branch has no errors when building and testing our client artifacts. +1 findbugs 1m 33s trunk passed +1 javadoc 0m 30s trunk passed       Patch Compile Tests +1 mvninstall 0m 42s the patch passed +1 compile 0m 37s the patch passed +1 javac 0m 37s the patch passed +1 checkstyle 0m 29s the patch passed +1 mvnsite 0m 41s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedclient 14m 18s patch has no errors when building and testing our client artifacts. +1 findbugs 1m 37s the patch passed +1 javadoc 0m 26s the patch passed       Other Tests -1 unit 45m 39s hadoop-yarn-server-resourcemanager in the patch passed. 0 asflicense 0m 32s ASF License check generated no output? 105m 43s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart Subsystem Report/Notes Docker Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 JIRA Issue YARN-10167 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12995313/YARN-10167-002.patch Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle uname Linux 32d186bade95 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision trunk / edc2e9d maven version: Apache Maven 3.3.9 Default Java 1.8.0_242 findbugs v3.1.0-RC1 unit https://builds.apache.org/job/PreCommit-YARN-Build/25615/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/25615/testReport/ Max. process+thread count 839 (vs. ulimit of 5500) modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/25615/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 37s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 8 new or modified test files.
                trunk Compile Tests
          +1 mvninstall 20m 9s trunk passed
          +1 compile 0m 43s trunk passed
          +1 checkstyle 0m 34s trunk passed
          +1 mvnsite 0m 45s trunk passed
          +1 shadedclient 15m 26s branch has no errors when building and testing our client artifacts.
          +1 findbugs 1m 36s trunk passed
          +1 javadoc 0m 30s trunk passed
                Patch Compile Tests
          +1 mvninstall 0m 43s the patch passed
          +1 compile 0m 40s the patch passed
          +1 javac 0m 40s the patch passed
          -0 checkstyle 0m 29s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 2 new + 3 unchanged - 0 fixed = 5 total (was 3)
          +1 mvnsite 0m 46s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 5s The patch has no ill-formed XML file.
          +1 shadedclient 14m 52s patch has no errors when building and testing our client artifacts.
          +1 findbugs 1m 56s the patch passed
          +1 javadoc 0m 28s the patch passed
                Other Tests
          -1 unit 91m 51s hadoop-yarn-server-resourcemanager in the patch passed.
          -1 asflicense 0m 28s The patch generated 5 ASF License warnings.
          152m 28s



          Reason Tests
          Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart



          Subsystem Report/Notes
          Docker Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3
          JIRA Issue YARN-10167
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12995461/YARN-10167-003.patch
          Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
          uname Linux f0ca7686ab6a 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/patchprocess/precommit/personality/provided.sh
          git revision trunk / c0d0842
          maven version: Apache Maven 3.3.9
          Default Java 1.8.0_242
          findbugs v3.1.0-RC1
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/25619/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/25619/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/25619/testReport/
          asflicense https://builds.apache.org/job/PreCommit-YARN-Build/25619/artifact/out/patch-asflicense-problems.txt
          Max. process+thread count 825 (vs. ulimit of 5500)
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/25619/console
          Powered by Apache Yetus 0.8.0 http://yetus.apache.org

          This message was automatically generated.

          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 37s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 8 new or modified test files.       trunk Compile Tests +1 mvninstall 20m 9s trunk passed +1 compile 0m 43s trunk passed +1 checkstyle 0m 34s trunk passed +1 mvnsite 0m 45s trunk passed +1 shadedclient 15m 26s branch has no errors when building and testing our client artifacts. +1 findbugs 1m 36s trunk passed +1 javadoc 0m 30s trunk passed       Patch Compile Tests +1 mvninstall 0m 43s the patch passed +1 compile 0m 40s the patch passed +1 javac 0m 40s the patch passed -0 checkstyle 0m 29s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 2 new + 3 unchanged - 0 fixed = 5 total (was 3) +1 mvnsite 0m 46s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 5s The patch has no ill-formed XML file. +1 shadedclient 14m 52s patch has no errors when building and testing our client artifacts. +1 findbugs 1m 56s the patch passed +1 javadoc 0m 28s the patch passed       Other Tests -1 unit 91m 51s hadoop-yarn-server-resourcemanager in the patch passed. -1 asflicense 0m 28s The patch generated 5 ASF License warnings. 152m 28s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart Subsystem Report/Notes Docker Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 JIRA Issue YARN-10167 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12995461/YARN-10167-003.patch Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml uname Linux f0ca7686ab6a 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision trunk / c0d0842 maven version: Apache Maven 3.3.9 Default Java 1.8.0_242 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/25619/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/25619/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/25619/testReport/ asflicense https://builds.apache.org/job/PreCommit-YARN-Build/25619/artifact/out/patch-asflicense-problems.txt Max. process+thread count 825 (vs. ulimit of 5500) modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/25619/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 43s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 8 new or modified test files.
                trunk Compile Tests
          -1 mvninstall 4m 52s root in trunk failed.
          -1 compile 0m 39s hadoop-yarn-server-resourcemanager in trunk failed.
          -0 checkstyle 0m 34s The patch fails to run checkstyle in hadoop-yarn-server-resourcemanager
          -1 mvnsite 0m 32s hadoop-yarn-server-resourcemanager in trunk failed.
          -1 shadedclient 1m 57s branch has errors when building and testing our client artifacts.
          -1 findbugs 0m 27s hadoop-yarn-server-resourcemanager in trunk failed.
          -1 javadoc 0m 27s hadoop-yarn-server-resourcemanager in trunk failed.
                Patch Compile Tests
          +1 mvninstall 1m 4s the patch passed
          +1 compile 0m 44s the patch passed
          -1 javac 0m 44s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager generated 27 new + 0 unchanged - 0 fixed = 27 total (was 0)
          -0 checkstyle 0m 30s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 5 new + 0 unchanged - 0 fixed = 5 total (was 0)
          +1 mvnsite 0m 45s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 5s The patch has no ill-formed XML file.
          +1 shadedclient 14m 9s patch has no errors when building and testing our client artifacts.
          +1 findbugs 1m 37s the patch passed
          -1 javadoc 0m 28s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager generated 69 new + 0 unchanged - 0 fixed = 69 total (was 0)
                Other Tests
          +1 unit 89m 30s hadoop-yarn-server-resourcemanager in the patch passed.
          +1 asflicense 0m 26s The patch does not generate ASF License warnings.
          119m 37s



          Subsystem Report/Notes
          Docker Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3
          JIRA Issue YARN-10167
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12995484/YARN-10167-004.patch
          Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
          uname Linux 3fe782050704 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/patchprocess/precommit/personality/provided.sh
          git revision trunk / c0d0842
          maven version: Apache Maven 3.3.9
          Default Java 1.8.0_242
          mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/25620/artifact/out/branch-mvninstall-root.txt
          compile https://builds.apache.org/job/PreCommit-YARN-Build/25620/artifact/out/branch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/25620/artifact/out//testptch/patchprocess/maven-branch-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          mvnsite https://builds.apache.org/job/PreCommit-YARN-Build/25620/artifact/out/branch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          findbugs https://builds.apache.org/job/PreCommit-YARN-Build/25620/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/25620/artifact/out/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          javac https://builds.apache.org/job/PreCommit-YARN-Build/25620/artifact/out/diff-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/25620/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/25620/artifact/out/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/25620/testReport/
          Max. process+thread count 819 (vs. ulimit of 5500)
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/25620/console
          Powered by Apache Yetus 0.8.0 http://yetus.apache.org

          This message was automatically generated.

          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 43s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 8 new or modified test files.       trunk Compile Tests -1 mvninstall 4m 52s root in trunk failed. -1 compile 0m 39s hadoop-yarn-server-resourcemanager in trunk failed. -0 checkstyle 0m 34s The patch fails to run checkstyle in hadoop-yarn-server-resourcemanager -1 mvnsite 0m 32s hadoop-yarn-server-resourcemanager in trunk failed. -1 shadedclient 1m 57s branch has errors when building and testing our client artifacts. -1 findbugs 0m 27s hadoop-yarn-server-resourcemanager in trunk failed. -1 javadoc 0m 27s hadoop-yarn-server-resourcemanager in trunk failed.       Patch Compile Tests +1 mvninstall 1m 4s the patch passed +1 compile 0m 44s the patch passed -1 javac 0m 44s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager generated 27 new + 0 unchanged - 0 fixed = 27 total (was 0) -0 checkstyle 0m 30s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 5 new + 0 unchanged - 0 fixed = 5 total (was 0) +1 mvnsite 0m 45s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 5s The patch has no ill-formed XML file. +1 shadedclient 14m 9s patch has no errors when building and testing our client artifacts. +1 findbugs 1m 37s the patch passed -1 javadoc 0m 28s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager generated 69 new + 0 unchanged - 0 fixed = 69 total (was 0)       Other Tests +1 unit 89m 30s hadoop-yarn-server-resourcemanager in the patch passed. +1 asflicense 0m 26s The patch does not generate ASF License warnings. 119m 37s Subsystem Report/Notes Docker Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 JIRA Issue YARN-10167 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12995484/YARN-10167-004.patch Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml uname Linux 3fe782050704 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision trunk / c0d0842 maven version: Apache Maven 3.3.9 Default Java 1.8.0_242 mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/25620/artifact/out/branch-mvninstall-root.txt compile https://builds.apache.org/job/PreCommit-YARN-Build/25620/artifact/out/branch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/25620/artifact/out//testptch/patchprocess/maven-branch-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt mvnsite https://builds.apache.org/job/PreCommit-YARN-Build/25620/artifact/out/branch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt findbugs https://builds.apache.org/job/PreCommit-YARN-Build/25620/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/25620/artifact/out/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt javac https://builds.apache.org/job/PreCommit-YARN-Build/25620/artifact/out/diff-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/25620/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/25620/artifact/out/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/25620/testReport/ Max. process+thread count 819 (vs. ulimit of 5500) modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/25620/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 39s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 8 new or modified test files.
                trunk Compile Tests
          +1 mvninstall 20m 25s trunk passed
          +1 compile 0m 49s trunk passed
          +1 checkstyle 0m 37s trunk passed
          +1 mvnsite 0m 56s trunk passed
          +1 shadedclient 16m 8s branch has no errors when building and testing our client artifacts.
          +1 findbugs 1m 35s trunk passed
          +1 javadoc 0m 31s trunk passed
                Patch Compile Tests
          +1 mvninstall 0m 48s the patch passed
          +1 compile 0m 44s the patch passed
          +1 javac 0m 44s the patch passed
          -0 checkstyle 0m 29s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 2 new + 3 unchanged - 0 fixed = 5 total (was 3)
          +1 mvnsite 0m 48s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 5s The patch has no ill-formed XML file.
          +1 shadedclient 14m 16s patch has no errors when building and testing our client artifacts.
          +1 findbugs 1m 55s the patch passed
          +1 javadoc 0m 32s the patch passed
                Other Tests
          +1 unit 89m 57s hadoop-yarn-server-resourcemanager in the patch passed.
          +1 asflicense 0m 30s The patch does not generate ASF License warnings.
          151m 28s



          Subsystem Report/Notes
          Docker Client=19.03.7 Server=19.03.7 Image:yetus/hadoop:c44943d1fc3
          JIRA Issue YARN-10167
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12995484/YARN-10167-004.patch
          Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
          uname Linux c1555f77b503 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/patchprocess/precommit/personality/provided.sh
          git revision trunk / bbd704b
          maven version: Apache Maven 3.3.9
          Default Java 1.8.0_242
          findbugs v3.1.0-RC1
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/25624/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/25624/testReport/
          Max. process+thread count 839 (vs. ulimit of 5500)
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/25624/console
          Powered by Apache Yetus 0.8.0 http://yetus.apache.org

          This message was automatically generated.

          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 39s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 8 new or modified test files.       trunk Compile Tests +1 mvninstall 20m 25s trunk passed +1 compile 0m 49s trunk passed +1 checkstyle 0m 37s trunk passed +1 mvnsite 0m 56s trunk passed +1 shadedclient 16m 8s branch has no errors when building and testing our client artifacts. +1 findbugs 1m 35s trunk passed +1 javadoc 0m 31s trunk passed       Patch Compile Tests +1 mvninstall 0m 48s the patch passed +1 compile 0m 44s the patch passed +1 javac 0m 44s the patch passed -0 checkstyle 0m 29s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 2 new + 3 unchanged - 0 fixed = 5 total (was 3) +1 mvnsite 0m 48s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 5s The patch has no ill-formed XML file. +1 shadedclient 14m 16s patch has no errors when building and testing our client artifacts. +1 findbugs 1m 55s the patch passed +1 javadoc 0m 32s the patch passed       Other Tests +1 unit 89m 57s hadoop-yarn-server-resourcemanager in the patch passed. +1 asflicense 0m 30s The patch does not generate ASF License warnings. 151m 28s Subsystem Report/Notes Docker Client=19.03.7 Server=19.03.7 Image:yetus/hadoop:c44943d1fc3 JIRA Issue YARN-10167 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12995484/YARN-10167-004.patch Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml uname Linux c1555f77b503 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision trunk / bbd704b maven version: Apache Maven 3.3.9 Default Java 1.8.0_242 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/25624/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/25624/testReport/ Max. process+thread count 839 (vs. ulimit of 5500) modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/25624/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
          pbacsko Peter Bacsko added a comment -

          snemeth could you review the latest v5 patch?

          pbacsko Peter Bacsko added a comment - snemeth could you review the latest v5 patch?
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 45s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 8 new or modified test files.
                trunk Compile Tests
          +1 mvninstall 20m 20s trunk passed
          +1 compile 0m 50s trunk passed
          +1 checkstyle 0m 38s trunk passed
          +1 mvnsite 0m 52s trunk passed
          +1 shadedclient 16m 8s branch has no errors when building and testing our client artifacts.
          +1 findbugs 1m 38s trunk passed
          +1 javadoc 0m 30s trunk passed
                Patch Compile Tests
          +1 mvninstall 0m 45s the patch passed
          +1 compile 0m 38s the patch passed
          +1 javac 0m 38s the patch passed
          +1 checkstyle 0m 28s the patch passed
          +1 mvnsite 0m 42s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 5s The patch has no ill-formed XML file.
          +1 shadedclient 15m 1s patch has no errors when building and testing our client artifacts.
          +1 findbugs 1m 45s the patch passed
          +1 javadoc 0m 28s the patch passed
                Other Tests
          +1 unit 89m 47s hadoop-yarn-server-resourcemanager in the patch passed.
          +1 asflicense 0m 28s The patch does not generate ASF License warnings.
          151m 23s



          Subsystem Report/Notes
          Docker Client=19.03.7 Server=19.03.7 Image:yetus/hadoop:c44943d1fc3
          JIRA Issue YARN-10167
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12995611/YARN-10167-005.patch
          Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
          uname Linux dc640b7e542e 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/patchprocess/precommit/personality/provided.sh
          git revision trunk / bbd704b
          maven version: Apache Maven 3.3.9
          Default Java 1.8.0_242
          findbugs v3.1.0-RC1
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/25627/testReport/
          Max. process+thread count 820 (vs. ulimit of 5500)
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/25627/console
          Powered by Apache Yetus 0.8.0 http://yetus.apache.org

          This message was automatically generated.

          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 45s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 8 new or modified test files.       trunk Compile Tests +1 mvninstall 20m 20s trunk passed +1 compile 0m 50s trunk passed +1 checkstyle 0m 38s trunk passed +1 mvnsite 0m 52s trunk passed +1 shadedclient 16m 8s branch has no errors when building and testing our client artifacts. +1 findbugs 1m 38s trunk passed +1 javadoc 0m 30s trunk passed       Patch Compile Tests +1 mvninstall 0m 45s the patch passed +1 compile 0m 38s the patch passed +1 javac 0m 38s the patch passed +1 checkstyle 0m 28s the patch passed +1 mvnsite 0m 42s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 5s The patch has no ill-formed XML file. +1 shadedclient 15m 1s patch has no errors when building and testing our client artifacts. +1 findbugs 1m 45s the patch passed +1 javadoc 0m 28s the patch passed       Other Tests +1 unit 89m 47s hadoop-yarn-server-resourcemanager in the patch passed. +1 asflicense 0m 28s The patch does not generate ASF License warnings. 151m 23s Subsystem Report/Notes Docker Client=19.03.7 Server=19.03.7 Image:yetus/hadoop:c44943d1fc3 JIRA Issue YARN-10167 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12995611/YARN-10167-005.patch Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml uname Linux dc640b7e542e 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision trunk / bbd704b maven version: Apache Maven 3.3.9 Default Java 1.8.0_242 findbugs v3.1.0-RC1 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/25627/testReport/ Max. process+thread count 820 (vs. ulimit of 5500) modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/25627/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org This message was automatically generated.
          snemeth Szilard Nemeth added a comment - - edited

          Hi pbacsko,
          Checked your patch, it looks good to me, committed it to trunk.
          One additional thing I did with your patch before committing is reformatting the XML configs.
          I think we need to update the description here, as it seems outdated.
          In my view, the reason why you did not use MockRM at all is you started up a CapacityScheduler instance (in ConvertedConfigValidator) and validate the config in a more "closer way" to production code.

          Can you remove / modify the description?
          I think this line should be changed:

          Also, in the test, we should try to leverage MockRM to validate generated configs as much as we could.

          snemeth Szilard Nemeth added a comment - - edited Hi pbacsko , Checked your patch, it looks good to me, committed it to trunk. One additional thing I did with your patch before committing is reformatting the XML configs. I think we need to update the description here, as it seems outdated. In my view, the reason why you did not use MockRM at all is you started up a CapacityScheduler instance (in ConvertedConfigValidator) and validate the config in a more "closer way" to production code. Can you remove / modify the description? I think this line should be changed: Also, in the test, we should try to leverage MockRM to validate generated configs as much as we could.
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #18028 (See https://builds.apache.org/job/Hadoop-trunk-Commit/18028/)
          YARN-10167. FS-CS Converter: Need to validate c-s.xml after converting. (snemeth: rev 004e95534817b3966d8e1727469c657cf1d043d7)

          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/FSConfigToCSConfigArgumentHandler.java
          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/DryRunResultHolder.java
          • (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/cs-validation-pass/yarn-site.xml
          • (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/VerificationException.java
          • (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/cs-validation-pass/capacity-scheduler.xml
          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/FSQueueConverter.java
          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/ConversionOptions.java
          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/TestFSConfigToCSConfigArgumentHandler.java
          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/TestFSConfigToCSConfigConverterMain.java
          • (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/cs-validation-fail/capacity-scheduler.xml
          • (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/cs-validation-fail/yarn-site.xml
          • (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/ConvertedConfigValidator.java
          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/TestFSQueueConverter.java
          • (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/TestConvertedConfigValidator.java
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #18028 (See https://builds.apache.org/job/Hadoop-trunk-Commit/18028/ ) YARN-10167 . FS-CS Converter: Need to validate c-s.xml after converting. (snemeth: rev 004e95534817b3966d8e1727469c657cf1d043d7) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/FSConfigToCSConfigArgumentHandler.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/DryRunResultHolder.java (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/cs-validation-pass/yarn-site.xml (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/VerificationException.java (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/cs-validation-pass/capacity-scheduler.xml (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/FSQueueConverter.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/ConversionOptions.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/TestFSConfigToCSConfigArgumentHandler.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/TestFSConfigToCSConfigConverterMain.java (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/cs-validation-fail/capacity-scheduler.xml (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/cs-validation-fail/yarn-site.xml (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/ConvertedConfigValidator.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/TestFSQueueConverter.java (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/TestConvertedConfigValidator.java

          People

            pbacsko Peter Bacsko
            leftnoteasy Wangda Tan
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: