Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-6339

Improve performance for createAndGetApplicationReport

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha4, 2.8.2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      There are two performance issue when calling createAndGetApplicationReport:
      One is inside ProtoUtils.convertFromProtoFormat, replace is too slow for clusters which have more than 3000 nodes. Use substring is much better: https://issues.apache.org/jira/browse/YARN-6285?focusedCommentId=15923241&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15923241

      Another one is inside getLogAggregationReportsForApp, if some application's LogAggregationStatus is TIME_OUT, every time it was called it will create an HashMap which will produce lots of garbage.

      1. YARN-6339.003.patch
        7 kB
        yunjiong zhao
      2. YARN-6339.002.patch
        8 kB
        yunjiong zhao
      3. YARN-6339.001.patch
        6 kB
        yunjiong zhao

        Issue Links

          Activity

          Hide
          zhaoyunjiong yunjiong zhao added a comment -

          This patch have 3 improvements:
          1. Use substring instead of replace
          2. Update logAggregationStatusForAppReport to reduce time spend in getLogAggregationStatusForAppReport.
          3. Inside getLogAggregationReportsForApp, move somecondition checks from for loop to outside, so for some applications, it won't run that for loop.

          Show
          zhaoyunjiong yunjiong zhao added a comment - This patch have 3 improvements: 1. Use substring instead of replace 2. Update logAggregationStatusForAppReport to reduce time spend in getLogAggregationStatusForAppReport. 3. Inside getLogAggregationReportsForApp, move somecondition checks from for loop to outside, so for some applications, it won't run that for loop.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 23s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 0m 48s Maven dependency ordering for branch
          +1 mvninstall 15m 30s trunk passed
          -1 compile 6m 9s hadoop-yarn in trunk failed.
          +1 checkstyle 1m 22s trunk passed
          +1 mvnsite 1m 50s trunk passed
          +1 mvneclipse 0m 53s trunk passed
          +1 findbugs 2m 48s trunk passed
          +1 javadoc 1m 27s trunk passed
          0 mvndep 0m 10s Maven dependency ordering for patch
          +1 mvninstall 1m 11s the patch passed
          -1 compile 5m 37s hadoop-yarn in the patch failed.
          -1 javac 5m 37s hadoop-yarn in the patch failed.
          -0 checkstyle 1m 9s hadoop-yarn-project/hadoop-yarn: The patch generated 2 new + 148 unchanged - 1 fixed = 150 total (was 149)
          +1 mvnsite 1m 45s the patch passed
          +1 mvneclipse 0m 57s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 3m 32s the patch passed
          +1 javadoc 1m 19s the patch passed
          +1 unit 2m 46s hadoop-yarn-common in the patch passed.
          -1 unit 42m 4s hadoop-yarn-server-resourcemanager in the patch failed.
          +1 asflicense 0m 45s The patch does not generate ASF License warnings.
          101m 44s



          Reason Tests
          Failed junit tests hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue YARN-6339
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12858791/YARN-6339.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 30aa69ee457b 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 4c66a8d
          Default Java 1.8.0_121
          compile https://builds.apache.org/job/PreCommit-YARN-Build/15276/artifact/patchprocess/branch-compile-hadoop-yarn-project_hadoop-yarn.txt
          findbugs v3.0.0
          compile https://builds.apache.org/job/PreCommit-YARN-Build/15276/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn.txt
          javac https://builds.apache.org/job/PreCommit-YARN-Build/15276/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/15276/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/15276/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15276/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/15276/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 23s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 48s Maven dependency ordering for branch +1 mvninstall 15m 30s trunk passed -1 compile 6m 9s hadoop-yarn in trunk failed. +1 checkstyle 1m 22s trunk passed +1 mvnsite 1m 50s trunk passed +1 mvneclipse 0m 53s trunk passed +1 findbugs 2m 48s trunk passed +1 javadoc 1m 27s trunk passed 0 mvndep 0m 10s Maven dependency ordering for patch +1 mvninstall 1m 11s the patch passed -1 compile 5m 37s hadoop-yarn in the patch failed. -1 javac 5m 37s hadoop-yarn in the patch failed. -0 checkstyle 1m 9s hadoop-yarn-project/hadoop-yarn: The patch generated 2 new + 148 unchanged - 1 fixed = 150 total (was 149) +1 mvnsite 1m 45s the patch passed +1 mvneclipse 0m 57s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 32s the patch passed +1 javadoc 1m 19s the patch passed +1 unit 2m 46s hadoop-yarn-common in the patch passed. -1 unit 42m 4s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 45s The patch does not generate ASF License warnings. 101m 44s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-6339 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12858791/YARN-6339.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 30aa69ee457b 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4c66a8d Default Java 1.8.0_121 compile https://builds.apache.org/job/PreCommit-YARN-Build/15276/artifact/patchprocess/branch-compile-hadoop-yarn-project_hadoop-yarn.txt findbugs v3.0.0 compile https://builds.apache.org/job/PreCommit-YARN-Build/15276/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn.txt javac https://builds.apache.org/job/PreCommit-YARN-Build/15276/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/15276/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/15276/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15276/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/15276/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          zhaoyunjiong yunjiong zhao added a comment -

          Update patch for more improvement.
          Change RMAppImpl.logAggregationStatus from HashMap to ConcurrentHashMap so even hold a readlock, we can safely update logAggregationStatus.
          Then return Collections.unmodifiableMap to avoid create too many HashMap.

          Show
          zhaoyunjiong yunjiong zhao added a comment - Update patch for more improvement. Change RMAppImpl.logAggregationStatus from HashMap to ConcurrentHashMap so even hold a readlock, we can safely update logAggregationStatus. Then return Collections.unmodifiableMap to avoid create too many HashMap.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 22s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 0m 10s Maven dependency ordering for branch
          +1 mvninstall 14m 11s trunk passed
          +1 compile 6m 12s trunk passed
          +1 checkstyle 0m 50s trunk passed
          +1 mvnsite 1m 16s trunk passed
          +1 mvneclipse 0m 39s trunk passed
          +1 findbugs 2m 8s trunk passed
          +1 javadoc 1m 1s trunk passed
          0 mvndep 0m 9s Maven dependency ordering for patch
          +1 mvninstall 0m 55s the patch passed
          +1 compile 5m 26s the patch passed
          +1 javac 5m 26s the patch passed
          -0 checkstyle 0m 49s hadoop-yarn-project/hadoop-yarn: The patch generated 2 new + 148 unchanged - 1 fixed = 150 total (was 149)
          +1 mvnsite 1m 14s the patch passed
          +1 mvneclipse 0m 38s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 2m 27s the patch passed
          +1 javadoc 0m 57s the patch passed
          +1 unit 2m 33s hadoop-yarn-common in the patch passed.
          +1 unit 42m 19s hadoop-yarn-server-resourcemanager in the patch passed.
          +1 asflicense 0m 37s The patch does not generate ASF License warnings.
          94m 5s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue YARN-6339
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12859186/YARN-6339.002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux ab1bd35c2c4a 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 4812518
          Default Java 1.8.0_121
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/15305/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15305/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/15305/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 10s Maven dependency ordering for branch +1 mvninstall 14m 11s trunk passed +1 compile 6m 12s trunk passed +1 checkstyle 0m 50s trunk passed +1 mvnsite 1m 16s trunk passed +1 mvneclipse 0m 39s trunk passed +1 findbugs 2m 8s trunk passed +1 javadoc 1m 1s trunk passed 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 0m 55s the patch passed +1 compile 5m 26s the patch passed +1 javac 5m 26s the patch passed -0 checkstyle 0m 49s hadoop-yarn-project/hadoop-yarn: The patch generated 2 new + 148 unchanged - 1 fixed = 150 total (was 149) +1 mvnsite 1m 14s the patch passed +1 mvneclipse 0m 38s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 27s the patch passed +1 javadoc 0m 57s the patch passed +1 unit 2m 33s hadoop-yarn-common in the patch passed. +1 unit 42m 19s hadoop-yarn-server-resourcemanager in the patch passed. +1 asflicense 0m 37s The patch does not generate ASF License warnings. 94m 5s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-6339 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12859186/YARN-6339.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux ab1bd35c2c4a 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4812518 Default Java 1.8.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/15305/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15305/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/15305/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Thanks yunjiong zhao,

          For your latest patch, there's only thing I'm not sure:

          Why changes of createAndGetApplicationReport required? It will be helpful if you could share more details about this part.

          Other changes of the patch looks good.

          Show
          leftnoteasy Wangda Tan added a comment - Thanks yunjiong zhao , For your latest patch, there's only thing I'm not sure: Why changes of createAndGetApplicationReport required? It will be helpful if you could share more details about this part. Other changes of the patch looks good.
          Hide
          zhaoyunjiong yunjiong zhao added a comment -

          Why changes of createAndGetApplicationReport required?

          The purpose is to avoid calling getLogAggregationStatus() unnecessary inside getLogAggregationReportsForApp() after application's LogAggregationStatus changed to TIME_OUT.
          I think we should add LogAggregationStatus.TIME_OUT in isLogAggregationFinished() like LogAggregationStatus.SUCCEEDED and LogAggregationStatus.FAILED.

          If ignore future risks, we can even change logAggregationStatusForAppReport inside getLogAggregationStatusForAppReport() with hold readLock only. To avoid confusing, due to createAndGetApplicationReport() will call getLogAggregationStatusForAppReport() with hold readLock, I think update logAggregationStatusForAppReport inside createAndGetApplicationReport() with writeLock hold is right thing to do.

                  } else if (logTimeOutCount > 0) {
          +         logAggregationStatusForAppReport = LogAggregationStatus.TIME_OUT; 
                    return LogAggregationStatus.TIME_OUT;
                  }
          
          Show
          zhaoyunjiong yunjiong zhao added a comment - Why changes of createAndGetApplicationReport required? The purpose is to avoid calling getLogAggregationStatus() unnecessary inside getLogAggregationReportsForApp() after application's LogAggregationStatus changed to TIME_OUT. I think we should add LogAggregationStatus.TIME_OUT in isLogAggregationFinished() like LogAggregationStatus.SUCCEEDED and LogAggregationStatus.FAILED. If ignore future risks, we can even change logAggregationStatusForAppReport inside getLogAggregationStatusForAppReport() with hold readLock only. To avoid confusing, due to createAndGetApplicationReport() will call getLogAggregationStatusForAppReport() with hold readLock, I think update logAggregationStatusForAppReport inside createAndGetApplicationReport() with writeLock hold is right thing to do. } else if (logTimeOutCount > 0) { + logAggregationStatusForAppReport = LogAggregationStatus.TIME_OUT; return LogAggregationStatus.TIME_OUT; }
          Hide
          leftnoteasy Wangda Tan added a comment - - edited

          yunjiong zhao, thanks for explanation, generally make sense to me.

          Few more suggestions:

          1) Can we make use of isLogAggregationFinished in createAndGetApplicationReport to avoid following checks:

                if (LogAggregationStatus.FAILED == logAggregationStatusApp ||
                    LogAggregationStatus.SUCCEEDED == logAggregationStatusApp ||
                    LogAggregationStatus.TIME_OUT == logAggregationStatusApp) {
                  try {
                    this.writeLock.lock();
                    this.logAggregationStatusForAppReport = logAggregationStatusApp;
                  } finally {
                    this.writeLock.unlock();
                  }
                }
          

          2) Can we make logAggregationStatusForAppReport to be volatile to avoid the additional writelock of createAndGetApplicationReport?

          Show
          leftnoteasy Wangda Tan added a comment - - edited yunjiong zhao , thanks for explanation, generally make sense to me. Few more suggestions: 1) Can we make use of isLogAggregationFinished in createAndGetApplicationReport to avoid following checks: if (LogAggregationStatus.FAILED == logAggregationStatusApp || LogAggregationStatus.SUCCEEDED == logAggregationStatusApp || LogAggregationStatus.TIME_OUT == logAggregationStatusApp) { try { this .writeLock.lock(); this .logAggregationStatusForAppReport = logAggregationStatusApp; } finally { this .writeLock.unlock(); } } 2) Can we make logAggregationStatusForAppReport to be volatile to avoid the additional writelock of createAndGetApplicationReport ?
          Hide
          zhaoyunjiong yunjiong zhao added a comment -

          Tan, Wangda, Good suggestion.
          Update patch set logAggregationStatusForAppReport to volatile, no need changes in createAndGetApplicationReport() any more since it's safe update logAggregationStatusForAppReport inside getLogAggregationStatusForAppReport().
          Thanks for your time to review the patch.

          Show
          zhaoyunjiong yunjiong zhao added a comment - Tan, Wangda , Good suggestion. Update patch set logAggregationStatusForAppReport to volatile, no need changes in createAndGetApplicationReport() any more since it's safe update logAggregationStatusForAppReport inside getLogAggregationStatusForAppReport(). Thanks for your time to review the patch.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 21s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 16s Maven dependency ordering for branch
          +1 mvninstall 15m 44s trunk passed
          +1 compile 18m 53s trunk passed
          +1 checkstyle 1m 3s trunk passed
          +1 mvnsite 1m 36s trunk passed
          +1 mvneclipse 0m 48s trunk passed
          +1 findbugs 2m 45s trunk passed
          +1 javadoc 1m 42s trunk passed
          0 mvndep 0m 25s Maven dependency ordering for patch
          -1 mvninstall 0m 40s hadoop-yarn-common in the patch failed.
          -1 mvninstall 0m 41s hadoop-yarn-server-resourcemanager in the patch failed.
          +1 compile 13m 11s the patch passed
          +1 javac 13m 11s the patch passed
          +1 checkstyle 0m 59s the patch passed
          +1 mvnsite 1m 26s the patch passed
          +1 mvneclipse 0m 43s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 3m 16s the patch passed
          +1 javadoc 1m 14s the patch passed
          +1 unit 2m 59s hadoop-yarn-common in the patch passed.
          +1 unit 41m 51s hadoop-yarn-server-resourcemanager in the patch passed.
          +1 asflicense 0m 33s The patch does not generate ASF License warnings.
          120m 1s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue YARN-6339
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12859809/YARN-6339.003.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 182c97830a1e 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / f462e1f
          Default Java 1.8.0_121
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/15352/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
          mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/15352/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15352/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/15352/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 16s Maven dependency ordering for branch +1 mvninstall 15m 44s trunk passed +1 compile 18m 53s trunk passed +1 checkstyle 1m 3s trunk passed +1 mvnsite 1m 36s trunk passed +1 mvneclipse 0m 48s trunk passed +1 findbugs 2m 45s trunk passed +1 javadoc 1m 42s trunk passed 0 mvndep 0m 25s Maven dependency ordering for patch -1 mvninstall 0m 40s hadoop-yarn-common in the patch failed. -1 mvninstall 0m 41s hadoop-yarn-server-resourcemanager in the patch failed. +1 compile 13m 11s the patch passed +1 javac 13m 11s the patch passed +1 checkstyle 0m 59s the patch passed +1 mvnsite 1m 26s the patch passed +1 mvneclipse 0m 43s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 16s the patch passed +1 javadoc 1m 14s the patch passed +1 unit 2m 59s hadoop-yarn-common in the patch passed. +1 unit 41m 51s hadoop-yarn-server-resourcemanager in the patch passed. +1 asflicense 0m 33s The patch does not generate ASF License warnings. 120m 1s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-6339 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12859809/YARN-6339.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 182c97830a1e 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f462e1f Default Java 1.8.0_121 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/15352/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/15352/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15352/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/15352/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Thanks for update the patch yunjiong zhao. Latest patch LGTM.

          cc: Xuan Gong, could you take a look at the patch as well?

          Show
          leftnoteasy Wangda Tan added a comment - Thanks for update the patch yunjiong zhao . Latest patch LGTM. cc: Xuan Gong , could you take a look at the patch as well?
          Hide
          xgong Xuan Gong added a comment -

          yunjiong zhao Thanks for the fix.
          +1 LGTM

          Show
          xgong Xuan Gong added a comment - yunjiong zhao Thanks for the fix. +1 LGTM
          Hide
          leftnoteasy Wangda Tan added a comment -

          Committing ..

          Show
          leftnoteasy Wangda Tan added a comment - Committing ..
          Hide
          leftnoteasy Wangda Tan added a comment -

          Committed to trunk/branch-2/branch-2.8, thanks yunjiong zhao and thanks reviews from Xuan Gong!

          Show
          leftnoteasy Wangda Tan added a comment - Committed to trunk/branch-2/branch-2.8, thanks yunjiong zhao and thanks reviews from Xuan Gong !
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11475 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11475/)
          YARN-6339. Improve performance for createAndGetApplicationReport. (wangda: rev cd014d57aa8b896da02b5bcadafbd404bca2bc12)

          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/logaggregationstatus/TestRMAppLogAggregationStatus.java
          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ProtoUtils.java
          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11475 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11475/ ) YARN-6339 . Improve performance for createAndGetApplicationReport. (wangda: rev cd014d57aa8b896da02b5bcadafbd404bca2bc12) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/logaggregationstatus/TestRMAppLogAggregationStatus.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ProtoUtils.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
          Hide
          zhaoyunjiong yunjiong zhao added a comment -

          Thanks Tan, Wangda & Xuan Gong for your time.

          Show
          zhaoyunjiong yunjiong zhao added a comment - Thanks Tan, Wangda & Xuan Gong for your time.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          2.8.1 became a security release. Moving fix-version to 2.8.2 after the fact.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - 2.8.1 became a security release. Moving fix-version to 2.8.2 after the fact.

            People

            • Assignee:
              zhaoyunjiong yunjiong zhao
              Reporter:
              zhaoyunjiong yunjiong zhao
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development