Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: log-aggregation
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Per discussions in YARN-1402, we shouldn't cache all node's log aggregation reports in RMApps for always, especially for those finished with SUCCEED.

      1. YARN-3505.addendum.patch
        1 kB
        Xuan Gong
      2. YARN-3505.6.patch
        71 kB
        Junping Du
      3. YARN-3505.5.patch
        71 kB
        Xuan Gong
      4. YARN-3505.4.patch
        69 kB
        Xuan Gong
      5. YARN-3505.3.patch
        68 kB
        Xuan Gong
      6. YARN-3505.2.rebase.patch
        65 kB
        Xuan Gong
      7. YARN-3505.2.patch
        65 kB
        Xuan Gong
      8. YARN-3505.1.patch
        68 kB
        Xuan Gong

        Issue Links

          Activity

          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 35s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace.
          +1 javac 7m 33s There were no new javac warning messages.
          +1 javadoc 9m 33s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 7m 36s The applied patch generated 13 additional checkstyle issues.
          +1 install 1m 35s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          -1 findbugs 3m 12s The patch appears to introduce 1 new Findbugs (version 2.0.3) warnings.
          +1 yarn tests 0m 24s Tests passed in hadoop-yarn-server-common.
          +1 yarn tests 5m 55s Tests passed in hadoop-yarn-server-nodemanager.
          +1 yarn tests 51m 41s Tests passed in hadoop-yarn-server-resourcemanager.
              103m 3s  



          Reason Tests
          FindBugs module:hadoop-yarn-server-resourcemanager
            org.apache.hadoop.yarn.server.resourcemanager.webapp.RMAppLogAggregationStatusBlock.render(HtmlBlock$Block) concatenates strings using + in a loop At RMAppLogAggregationStatusBlock.java:in a loop At RMAppLogAggregationStatusBlock.java:[line 136]



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12727028/YARN-3505.1.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 2c14690
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/7437/artifact/patchprocess/whitespace.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7437/artifact/patchprocess/checkstyle-result-diff.txt
          Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/7437/artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
          hadoop-yarn-server-common test log https://builds.apache.org/job/PreCommit-YARN-Build/7437/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt
          hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7437/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7437/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7437/testReport/
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/7437//console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 35s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. +1 javac 7m 33s There were no new javac warning messages. +1 javadoc 9m 33s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 7m 36s The applied patch generated 13 additional checkstyle issues. +1 install 1m 35s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. -1 findbugs 3m 12s The patch appears to introduce 1 new Findbugs (version 2.0.3) warnings. +1 yarn tests 0m 24s Tests passed in hadoop-yarn-server-common. +1 yarn tests 5m 55s Tests passed in hadoop-yarn-server-nodemanager. +1 yarn tests 51m 41s Tests passed in hadoop-yarn-server-resourcemanager.     103m 3s   Reason Tests FindBugs module:hadoop-yarn-server-resourcemanager   org.apache.hadoop.yarn.server.resourcemanager.webapp.RMAppLogAggregationStatusBlock.render(HtmlBlock$Block) concatenates strings using + in a loop At RMAppLogAggregationStatusBlock.java:in a loop At RMAppLogAggregationStatusBlock.java: [line 136] Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12727028/YARN-3505.1.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 2c14690 whitespace https://builds.apache.org/job/PreCommit-YARN-Build/7437/artifact/patchprocess/whitespace.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7437/artifact/patchprocess/checkstyle-result-diff.txt Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/7437/artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html hadoop-yarn-server-common test log https://builds.apache.org/job/PreCommit-YARN-Build/7437/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7437/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7437/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7437/testReport/ Console output https://builds.apache.org/job/PreCommit-YARN-Build/7437//console This message was automatically generated.
          Hide
          djp Junping Du added a comment -

          Thanks for uploading a patch to fix this problem, Xuan Gong! I am reviewing your patch. In the mean time, can you fix the findbugs warning message which seems to be related with the code?

          Show
          djp Junping Du added a comment - Thanks for uploading a patch to fix this problem, Xuan Gong ! I am reviewing your patch. In the mean time, can you fix the findbugs warning message which seems to be related with the code?
          Hide
          djp Junping Du added a comment -

          Sorry for comments a little late. Some initiative comments/questions after go though the patch quickly:

          • Why we need LogAggregationReportInNM which is the same as LogAggregationReport? Is this just duplicated with LogAggregationReport?
          • Why we need to differentiate diagnosticMessage with failureMessages in LogAggregationReport? If we don't want to cache successful report, we can differentiate from LogAggregationStatus. Isn't it?
          Show
          djp Junping Du added a comment - Sorry for comments a little late. Some initiative comments/questions after go though the patch quickly: Why we need LogAggregationReportInNM which is the same as LogAggregationReport? Is this just duplicated with LogAggregationReport? Why we need to differentiate diagnosticMessage with failureMessages in LogAggregationReport? If we don't want to cache successful report, we can differentiate from LogAggregationStatus. Isn't it? From Vinod's comments in YARN-3445 ( https://issues.apache.org/jira/browse/YARN-3445?focusedCommentId=14520396&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14520396 ), we have duplicated applicationID info for logAggregationReportsForApps, so I think we replace map (appID -> logAggregationReport) with list of logAggregationReport (or a SET with override equalsTo with comparing applicationID only).
          Hide
          jianhe Jian He added a comment -

          One other thing, the LogAggregationReport#(get/set)getNodeId can also be removed as it's not used anywhere.
          I'm also unsure about the usage of LogAggregationReport#(get/set)DiagnosticMessage as it's only set with an empty string.

          Show
          jianhe Jian He added a comment - One other thing, the LogAggregationReport#(get/set)getNodeId can also be removed as it's not used anywhere. I'm also unsure about the usage of LogAggregationReport#(get/set)DiagnosticMessage as it's only set with an empty string.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Canceling the patch to address review comments.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Canceling the patch to address review comments.
          Hide
          xgong Xuan Gong added a comment -

          Addressed all comments. Junping Du, please review.

          Show
          xgong Xuan Gong added a comment - Addressed all comments. Junping Du , please review.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 37s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 30s There were no new javac warning messages.
          +1 javadoc 9m 36s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 13s The applied patch generated 4 new checkstyle issues (total was 71, now 66).
          -1 whitespace 0m 21s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 37s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 3m 9s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 yarn tests 0m 25s Tests passed in hadoop-yarn-server-common.
          +1 yarn tests 5m 50s Tests passed in hadoop-yarn-server-nodemanager.
          -1 yarn tests 62m 48s Tests failed in hadoop-yarn-server-resourcemanager.
              108m 28s  



          Reason Tests
          Timed out tests org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12731082/YARN-3505.2.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 918af8e
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7755/artifact/patchprocess/diffcheckstylehadoop-yarn-server-common.txt
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/7755/artifact/patchprocess/whitespace.txt
          hadoop-yarn-server-common test log https://builds.apache.org/job/PreCommit-YARN-Build/7755/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt
          hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7755/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7755/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7755/testReport/
          Java 1.7.0_55
          uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/7755/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 37s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 30s There were no new javac warning messages. +1 javadoc 9m 36s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 13s The applied patch generated 4 new checkstyle issues (total was 71, now 66). -1 whitespace 0m 21s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 37s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 3m 9s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 yarn tests 0m 25s Tests passed in hadoop-yarn-server-common. +1 yarn tests 5m 50s Tests passed in hadoop-yarn-server-nodemanager. -1 yarn tests 62m 48s Tests failed in hadoop-yarn-server-resourcemanager.     108m 28s   Reason Tests Timed out tests org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12731082/YARN-3505.2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 918af8e checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7755/artifact/patchprocess/diffcheckstylehadoop-yarn-server-common.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/7755/artifact/patchprocess/whitespace.txt hadoop-yarn-server-common test log https://builds.apache.org/job/PreCommit-YARN-Build/7755/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7755/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7755/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7755/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/7755/console This message was automatically generated.
          Hide
          djp Junping Du added a comment -

          Thanks for updating the patch, Xuan Gong! I will take a look soon.

          Show
          djp Junping Du added a comment - Thanks for updating the patch, Xuan Gong ! I will take a look soon.
          Hide
          xgong Xuan Gong added a comment -

          rebase the patch

          Show
          xgong Xuan Gong added a comment - rebase the patch
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 15m 7s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 41s There were no new javac warning messages.
          +1 javadoc 9m 50s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 16s The applied patch generated 3 new checkstyle issues (total was 70, now 64).
          +1 whitespace 0m 17s The patch has no lines that end in whitespace.
          +1 install 1m 38s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 3m 12s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 yarn tests 0m 24s Tests passed in hadoop-yarn-server-common.
          -1 yarn tests 5m 58s Tests failed in hadoop-yarn-server-nodemanager.
          -1 yarn tests 63m 52s Tests failed in hadoop-yarn-server-resourcemanager.
              110m 37s  



          Reason Tests
          Failed unit tests hadoop.yarn.server.nodemanager.webapp.TestNMWebServicesApps
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          Timed out tests org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12731215/YARN-3505.2.rebase.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 8e991f4
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7764/artifact/patchprocess/diffcheckstylehadoop-yarn-server-common.txt
          hadoop-yarn-server-common test log https://builds.apache.org/job/PreCommit-YARN-Build/7764/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt
          hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7764/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7764/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7764/testReport/
          Java 1.7.0_55
          uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/7764/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 15m 7s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 41s There were no new javac warning messages. +1 javadoc 9m 50s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 16s The applied patch generated 3 new checkstyle issues (total was 70, now 64). +1 whitespace 0m 17s The patch has no lines that end in whitespace. +1 install 1m 38s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 3m 12s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 yarn tests 0m 24s Tests passed in hadoop-yarn-server-common. -1 yarn tests 5m 58s Tests failed in hadoop-yarn-server-nodemanager. -1 yarn tests 63m 52s Tests failed in hadoop-yarn-server-resourcemanager.     110m 37s   Reason Tests Failed unit tests hadoop.yarn.server.nodemanager.webapp.TestNMWebServicesApps   hadoop.yarn.server.resourcemanager.TestAMAuthorization Timed out tests org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12731215/YARN-3505.2.rebase.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 8e991f4 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7764/artifact/patchprocess/diffcheckstylehadoop-yarn-server-common.txt hadoop-yarn-server-common test log https://builds.apache.org/job/PreCommit-YARN-Build/7764/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7764/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7764/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7764/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/7764/console This message was automatically generated.
          Hide
          djp Junping Du added a comment -

          Thanks Xuan for updating the patch! The latest patch looks much closer.

          Why we need to differentiate diagnosticMessage with failureMessages in LogAggregationReport? If we don't want to cache successful report, we can differentiate from LogAggregationStatus. Isn't it?

          It seems this previous comment haven't been responsed so far. Just think it again, I think our intention is trying to differentiate the normal diagnosticMessage and errorMessages, so it should be fine to track LogAggregationDiagnosticsForNMs and LogAggregationFailureMessagesForNMs separatedly in RMAppImpl. However, I think we don't need this differentiation in LogAggregationReport as in case of FAILED case, diagnosticMessage and failureMessages store the same message which is duplicated. Isn't it?

          Other minor comments:

          In RMAppImpl.java,
          aggregateLogReport() method sounds too long and need some refactor work here. Can we abstract some methods like: updateLogAggregationStatus(), updateLogAggregationDiagnosticMessages(), updateLogAggregationFailureMessages(), etc.? In addition, what would happen if (this.logAggregationSucceed + this.logAggregationFailed) != this.logAggregationStatus.size()?

          private static int MAX_LOG_AGGREGATION_DIAGNOSTICS_IN_MEMORY = 10;
          

          Shall we make it configurable? May be as private first (not putting on yarn-default.xml)?

          In AppBlock.java,

          +  protected LogAggregationStatus getLogAggregationStatus() {
          +    return null;
          +  }
          

          Can we add a comment to said it has to be overriden in subclass?

          +        th(_TH, "Last 10 Diagnostis Messages").
          

          Typo, "Diagnostis" => "Diagnostic"

          Show
          djp Junping Du added a comment - Thanks Xuan for updating the patch! The latest patch looks much closer. Why we need to differentiate diagnosticMessage with failureMessages in LogAggregationReport? If we don't want to cache successful report, we can differentiate from LogAggregationStatus. Isn't it? It seems this previous comment haven't been responsed so far. Just think it again, I think our intention is trying to differentiate the normal diagnosticMessage and errorMessages, so it should be fine to track LogAggregationDiagnosticsForNMs and LogAggregationFailureMessagesForNMs separatedly in RMAppImpl. However, I think we don't need this differentiation in LogAggregationReport as in case of FAILED case, diagnosticMessage and failureMessages store the same message which is duplicated. Isn't it? Other minor comments: In RMAppImpl.java, aggregateLogReport() method sounds too long and need some refactor work here. Can we abstract some methods like: updateLogAggregationStatus(), updateLogAggregationDiagnosticMessages(), updateLogAggregationFailureMessages(), etc.? In addition, what would happen if (this.logAggregationSucceed + this.logAggregationFailed) != this.logAggregationStatus.size()? private static int MAX_LOG_AGGREGATION_DIAGNOSTICS_IN_MEMORY = 10; Shall we make it configurable? May be as private first (not putting on yarn-default.xml)? In AppBlock.java, + protected LogAggregationStatus getLogAggregationStatus() { + return null ; + } Can we add a comment to said it has to be overriden in subclass? + th(_TH, "Last 10 Diagnostis Messages" ). Typo, "Diagnostis" => "Diagnostic"
          Hide
          djp Junping Du added a comment -

          However, I think we don't need this differentiation in LogAggregationReport as in case of FAILED case, diagnosticMessage and failureMessages store the same message which is duplicated. Isn't it?

          Just sync offline with Xuan Gong, the additional challenge here is for RUNNING LogAggregationState, we cannot 100% sure the log aggregation on this node is successful or failed (LRS need aggregate log for several times, we always make it RUNNING until final state). The proposed solution here is we may add a new state here called RUNNING_WITH_FAILURE for this case, so we differentiate the message is normal diagnostic one or failure messages. The extra bonus is user can know something wrong happens to LogAggregation earlier but not get confused with Log Aggregation already get stopped.

          Show
          djp Junping Du added a comment - However, I think we don't need this differentiation in LogAggregationReport as in case of FAILED case, diagnosticMessage and failureMessages store the same message which is duplicated. Isn't it? Just sync offline with Xuan Gong , the additional challenge here is for RUNNING LogAggregationState, we cannot 100% sure the log aggregation on this node is successful or failed (LRS need aggregate log for several times, we always make it RUNNING until final state). The proposed solution here is we may add a new state here called RUNNING_WITH_FAILURE for this case, so we differentiate the message is normal diagnostic one or failure messages. The extra bonus is user can know something wrong happens to LogAggregation earlier but not get confused with Log Aggregation already get stopped.
          Hide
          xgong Xuan Gong added a comment -

          In addition, what would happen if (this.logAggregationSucceed + this.logAggregationFailed) != this.logAggregationStatus.size()?

          If this happens, that means the log aggregation still happens in some of NMs.

          Upload a new patch to address other comments

          Show
          xgong Xuan Gong added a comment - In addition, what would happen if (this.logAggregationSucceed + this.logAggregationFailed) != this.logAggregationStatus.size()? If this happens, that means the log aggregation still happens in some of NMs. Upload a new patch to address other comments
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 15m 4s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 42s There were no new javac warning messages.
          +1 javadoc 10m 4s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 50s The applied patch generated 1 new checkstyle issues (total was 1, now 2).
          -1 checkstyle 2m 7s The applied patch generated 2 new checkstyle issues (total was 70, now 63).
          +1 whitespace 0m 19s The patch has no lines that end in whitespace.
          +1 install 1m 43s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 4m 34s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 yarn tests 0m 30s Tests passed in hadoop-yarn-api.
          +1 yarn tests 0m 23s Tests passed in hadoop-yarn-server-common.
          -1 yarn tests 5m 53s Tests failed in hadoop-yarn-server-nodemanager.
          -1 yarn tests 27m 33s Tests failed in hadoop-yarn-server-resourcemanager.
              77m 15s  



          Reason Tests
          Failed unit tests hadoop.yarn.server.nodemanager.containermanager.logaggregation.TestLogAggregationService
            hadoop.yarn.server.resourcemanager.security.TestAMRMTokens
            hadoop.yarn.server.resourcemanager.TestResourceTrackerService
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestWorkPreservingRMRestartForNodeLabel
            hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
            hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRMRPCNodeUpdates
            hadoop.yarn.server.resourcemanager.TestResourceManager
            hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRMRPCResponseId
            hadoop.yarn.server.resourcemanager.TestRM
            hadoop.yarn.server.resourcemanager.TestApplicationMasterService
            hadoop.yarn.server.resourcemanager.TestClientRMService
            hadoop.yarn.server.resourcemanager.webapp.TestAppPage
            hadoop.yarn.server.resourcemanager.resourcetracker.TestRMNMRPCResponseId
            hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
            hadoop.yarn.server.resourcemanager.logaggregationstatus.TestRMAppLogAggregationStatus
            hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
            hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens
            hadoop.yarn.server.resourcemanager.TestRMRestart
            hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesHttpStaticUserPermissions
            hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter
            hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerUtils
            hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA
            hadoop.yarn.server.resourcemanager.TestApplicationCleanup
            hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler
            hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNodeLabelUpdate



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12731665/YARN-3505.3.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 6471d18
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7836/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt https://builds.apache.org/job/PreCommit-YARN-Build/7836/artifact/patchprocess/diffcheckstylehadoop-yarn-server-common.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/7836/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-server-common test log https://builds.apache.org/job/PreCommit-YARN-Build/7836/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt
          hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7836/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7836/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7836/testReport/
          Java 1.7.0_55
          uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/7836/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 15m 4s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 42s There were no new javac warning messages. +1 javadoc 10m 4s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 50s The applied patch generated 1 new checkstyle issues (total was 1, now 2). -1 checkstyle 2m 7s The applied patch generated 2 new checkstyle issues (total was 70, now 63). +1 whitespace 0m 19s The patch has no lines that end in whitespace. +1 install 1m 43s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 4m 34s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 yarn tests 0m 30s Tests passed in hadoop-yarn-api. +1 yarn tests 0m 23s Tests passed in hadoop-yarn-server-common. -1 yarn tests 5m 53s Tests failed in hadoop-yarn-server-nodemanager. -1 yarn tests 27m 33s Tests failed in hadoop-yarn-server-resourcemanager.     77m 15s   Reason Tests Failed unit tests hadoop.yarn.server.nodemanager.containermanager.logaggregation.TestLogAggregationService   hadoop.yarn.server.resourcemanager.security.TestAMRMTokens   hadoop.yarn.server.resourcemanager.TestResourceTrackerService   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestWorkPreservingRMRestartForNodeLabel   hadoop.yarn.server.resourcemanager.TestContainerResourceUsage   hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRMRPCNodeUpdates   hadoop.yarn.server.resourcemanager.TestResourceManager   hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRMRPCResponseId   hadoop.yarn.server.resourcemanager.TestRM   hadoop.yarn.server.resourcemanager.TestApplicationMasterService   hadoop.yarn.server.resourcemanager.TestClientRMService   hadoop.yarn.server.resourcemanager.webapp.TestAppPage   hadoop.yarn.server.resourcemanager.resourcetracker.TestRMNMRPCResponseId   hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation   hadoop.yarn.server.resourcemanager.logaggregationstatus.TestRMAppLogAggregationStatus   hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart   hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens   hadoop.yarn.server.resourcemanager.TestRMRestart   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesHttpStaticUserPermissions   hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter   hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerUtils   hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA   hadoop.yarn.server.resourcemanager.TestApplicationCleanup   hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler   hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNodeLabelUpdate Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12731665/YARN-3505.3.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 6471d18 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7836/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt https://builds.apache.org/job/PreCommit-YARN-Build/7836/artifact/patchprocess/diffcheckstylehadoop-yarn-server-common.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/7836/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-server-common test log https://builds.apache.org/job/PreCommit-YARN-Build/7836/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7836/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7836/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7836/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/7836/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 35s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 30s There were no new javac warning messages.
          +1 javadoc 9m 36s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 50s The applied patch generated 1 new checkstyle issues (total was 1, now 2).
          -1 checkstyle 2m 7s The applied patch generated 2 new checkstyle issues (total was 71, now 64).
          +1 whitespace 0m 25s The patch has no lines that end in whitespace.
          +1 install 1m 38s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 4m 30s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 yarn tests 0m 21s Tests passed in hadoop-yarn-api.
          +1 yarn tests 0m 25s Tests passed in hadoop-yarn-server-common.
          +1 yarn tests 5m 59s Tests passed in hadoop-yarn-server-nodemanager.
          +1 yarn tests 52m 13s Tests passed in hadoop-yarn-server-resourcemanager.
              100m 40s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12731665/YARN-3505.3.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 6471d18
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7845/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt https://builds.apache.org/job/PreCommit-YARN-Build/7845/artifact/patchprocess/diffcheckstylehadoop-yarn-server-common.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/7845/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-server-common test log https://builds.apache.org/job/PreCommit-YARN-Build/7845/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt
          hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7845/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7845/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7845/testReport/
          Java 1.7.0_55
          uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/7845/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 35s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 30s There were no new javac warning messages. +1 javadoc 9m 36s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 50s The applied patch generated 1 new checkstyle issues (total was 1, now 2). -1 checkstyle 2m 7s The applied patch generated 2 new checkstyle issues (total was 71, now 64). +1 whitespace 0m 25s The patch has no lines that end in whitespace. +1 install 1m 38s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 4m 30s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 yarn tests 0m 21s Tests passed in hadoop-yarn-api. +1 yarn tests 0m 25s Tests passed in hadoop-yarn-server-common. +1 yarn tests 5m 59s Tests passed in hadoop-yarn-server-nodemanager. +1 yarn tests 52m 13s Tests passed in hadoop-yarn-server-resourcemanager.     100m 40s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12731665/YARN-3505.3.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 6471d18 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7845/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt https://builds.apache.org/job/PreCommit-YARN-Build/7845/artifact/patchprocess/diffcheckstylehadoop-yarn-server-common.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/7845/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-server-common test log https://builds.apache.org/job/PreCommit-YARN-Build/7845/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7845/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7845/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7845/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/7845/console This message was automatically generated.
          Hide
          djp Junping Du added a comment -

          If this happens, that means the log aggregation still happens in some of NMs.

          I see. Agree that we don't need to do any cleanup in this case.
          Some minor comments on updated patch:

          In aggregateLogReport.java,

          -          if (report.getDiagnosticMessage() != null
          -              && !report.getDiagnosticMessage().isEmpty()) {
          -            curReport
          -              .setDiagnosticMessage(curReport.getDiagnosticMessage() == null
          -                  ? report.getDiagnosticMessage() : curReport
          -                    .getDiagnosticMessage() + report.getDiagnosticMessage());
          +            if (!curReport.getLogAggregationStatus().equals(
          +              LogAggregationStatus.SUCCEEDED)
          +                && !curReport.getLogAggregationStatus().equals(
          +                  LogAggregationStatus.FAILED)
          +                && (report.getLogAggregationStatus().equals(
          +                  LogAggregationStatus.SUCCEEDED)
          +                || report.getLogAggregationStatus().equals(
          +                  LogAggregationStatus.FAILED))) {
          +              statusChanged = true; // anchor 1 for comments
          +            }
          +            if (report.getLogAggregationStatus() != LogAggregationStatus.RUNNING
          +                || curReport.getLogAggregationStatus() !=
          +                    LogAggregationStatus.RUNNING_WITH_FAILURE) {
          +              curReport.setLogAggregationStatus(report
          +                .getLogAggregationStatus()); // anchor 2 for comments
          +            }
          

          Are we missing curReport.setLogAggregationStatus() (in above anchor 1 place)? We should set SUCCEEDED or FAILED to curReport. Isn't it? In addition, why we don't put statusChanged in above anchor 2 place? If we think statusChanged only hint status move to final state (SUCCEEDED or FAILED), then we should rename statusChanged to something like stateChangedToFinal which sounds more obviously.
          BTW, can we make logic here to be something simpler to make status get updated except only two cases?:
          1. curReport.getLogAggregationStatus() = report.getLogAggregationStatus();
          2. curReport.getLogAggregationStatus() = RUNNING_WITH_FAILURE && report.getLogAggregationStatus() = RUNNING

          In updateLogAggregationDiagnosticMessages(),

          if (report.getLogAggregationStatus()
          +          == LogAggregationStatus.RUNNING || report.getLogAggregationStatus()
          +          == LogAggregationStatus.SUCCEEDED || report.getLogAggregationStatus()
          +          == LogAggregationStatus.FAILED) {
          

          Why case of "report.getLogAggregationStatus() == LogAggregationStatus.FAILED" doesn't go to the other branch like: LogAggregationStatus.RUNNING_WITH_FAILURE?

          +        LogAggregationDiagnosticsForNMs.put(nodeId, diagnostics);
          

          Move this into block of "diagnostics == null", right after " diagnostics = new ArrayList<String>();", because we only need to call this the first time we put diagnostics info. The same problem for failureMessages too.

          Show
          djp Junping Du added a comment - If this happens, that means the log aggregation still happens in some of NMs. I see. Agree that we don't need to do any cleanup in this case. Some minor comments on updated patch: In aggregateLogReport.java, - if (report.getDiagnosticMessage() != null - && !report.getDiagnosticMessage().isEmpty()) { - curReport - .setDiagnosticMessage(curReport.getDiagnosticMessage() == null - ? report.getDiagnosticMessage() : curReport - .getDiagnosticMessage() + report.getDiagnosticMessage()); + if (!curReport.getLogAggregationStatus().equals( + LogAggregationStatus.SUCCEEDED) + && !curReport.getLogAggregationStatus().equals( + LogAggregationStatus.FAILED) + && (report.getLogAggregationStatus().equals( + LogAggregationStatus.SUCCEEDED) + || report.getLogAggregationStatus().equals( + LogAggregationStatus.FAILED))) { + statusChanged = true ; // anchor 1 for comments + } + if (report.getLogAggregationStatus() != LogAggregationStatus.RUNNING + || curReport.getLogAggregationStatus() != + LogAggregationStatus.RUNNING_WITH_FAILURE) { + curReport.setLogAggregationStatus(report + .getLogAggregationStatus()); // anchor 2 for comments + } Are we missing curReport.setLogAggregationStatus() (in above anchor 1 place)? We should set SUCCEEDED or FAILED to curReport. Isn't it? In addition, why we don't put statusChanged in above anchor 2 place? If we think statusChanged only hint status move to final state (SUCCEEDED or FAILED), then we should rename statusChanged to something like stateChangedToFinal which sounds more obviously. BTW, can we make logic here to be something simpler to make status get updated except only two cases?: 1. curReport.getLogAggregationStatus() = report.getLogAggregationStatus(); 2. curReport.getLogAggregationStatus() = RUNNING_WITH_FAILURE && report.getLogAggregationStatus() = RUNNING In updateLogAggregationDiagnosticMessages(), if (report.getLogAggregationStatus() + == LogAggregationStatus.RUNNING || report.getLogAggregationStatus() + == LogAggregationStatus.SUCCEEDED || report.getLogAggregationStatus() + == LogAggregationStatus.FAILED) { Why case of "report.getLogAggregationStatus() == LogAggregationStatus.FAILED" doesn't go to the other branch like: LogAggregationStatus.RUNNING_WITH_FAILURE? + LogAggregationDiagnosticsForNMs.put(nodeId, diagnostics); Move this into block of "diagnostics == null", right after " diagnostics = new ArrayList<String>();", because we only need to call this the first time we put diagnostics info. The same problem for failureMessages too.
          Hide
          xgong Xuan Gong added a comment -

          Upload a new patch to address all the comments

          Show
          xgong Xuan Gong added a comment - Upload a new patch to address all the comments
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 15m 12s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 50s There were no new javac warning messages.
          +1 javadoc 9m 51s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 2m 7s The applied patch generated 1 new checkstyle issues (total was 1, now 2).
          -1 checkstyle 2m 22s The applied patch generated 2 new checkstyle issues (total was 70, now 63).
          +1 whitespace 0m 21s The patch has no lines that end in whitespace.
          +1 install 1m 41s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 4m 35s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 yarn tests 0m 21s Tests passed in hadoop-yarn-api.
          +1 yarn tests 0m 24s Tests passed in hadoop-yarn-server-common.
          +1 yarn tests 6m 10s Tests passed in hadoop-yarn-server-nodemanager.
          -1 yarn tests 51m 55s Tests failed in hadoop-yarn-server-resourcemanager.
              102m 7s  



          Reason Tests
          Failed unit tests hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRMRPCResponseId
            hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
            hadoop.yarn.server.resourcemanager.TestAMAuthorization



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12732030/YARN-3505.4.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / ea11590
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7866/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt https://builds.apache.org/job/PreCommit-YARN-Build/7866/artifact/patchprocess/diffcheckstylehadoop-yarn-server-common.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/7866/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-server-common test log https://builds.apache.org/job/PreCommit-YARN-Build/7866/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt
          hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7866/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7866/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7866/testReport/
          Java 1.7.0_55
          uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/7866/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 15m 12s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 50s There were no new javac warning messages. +1 javadoc 9m 51s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 7s The applied patch generated 1 new checkstyle issues (total was 1, now 2). -1 checkstyle 2m 22s The applied patch generated 2 new checkstyle issues (total was 70, now 63). +1 whitespace 0m 21s The patch has no lines that end in whitespace. +1 install 1m 41s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 4m 35s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 yarn tests 0m 21s Tests passed in hadoop-yarn-api. +1 yarn tests 0m 24s Tests passed in hadoop-yarn-server-common. +1 yarn tests 6m 10s Tests passed in hadoop-yarn-server-nodemanager. -1 yarn tests 51m 55s Tests failed in hadoop-yarn-server-resourcemanager.     102m 7s   Reason Tests Failed unit tests hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRMRPCResponseId   hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12732030/YARN-3505.4.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / ea11590 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7866/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt https://builds.apache.org/job/PreCommit-YARN-Build/7866/artifact/patchprocess/diffcheckstylehadoop-yarn-server-common.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/7866/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-server-common test log https://builds.apache.org/job/PreCommit-YARN-Build/7866/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7866/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7866/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7866/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/7866/console This message was automatically generated.
          Hide
          djp Junping Du added a comment -

          Thanks Xuan Gong for updating the patch! Latest patch get much closer.

          +  private static final String MAX_LOG_AGGREGATION_DIAGNOSTICS_IN_MEMORY =
          +      YarnConfiguration.RM_PREFIX + "max-log-aggregation-diagnostics-in-memory";
          +  private static int DEFAULT_MAX_LOG_AGGREGATION_DIAGNOSTICS_IN_MEMORY = 10;
          

          Do we want to expose this configuration to user? If so, we should put it in yarn-default.xml with properly description on it.

          +            // If the log aggregation status got from latest nm heartbeat
          +            // is Running, and current log aggregation status is TimeOut,
          +            // based on whether there are any failure messages for this NM,
          +            // we will reset the log aggregation status as RUNNING or
          +            // RUNNING_WITH_FAILURE
          +              if (logAggregationFailureMessagesForNMs.get(nodeId) != null
          +                  && !logAggregationFailureMessagesForNMs.get(nodeId).isEmpty()) {
          +                curReport
          +                  .setLogAggregationStatus(LogAggregationStatus.RUNNING_WITH_FAILURE);
          +              }
          +            }
          +            curReport.setLogAggregationStatus(report
          +              .getLogAggregationStatus());
          

          "curReport.setLogAggregationStatus(LogAggregationStatus.RUNNING_WITH_FAILURE); " should be replaced by "report.setLogAggregationStatus(..." ? Or it could be override later by RUNNING.

          Other looks fine to me.

          Show
          djp Junping Du added a comment - Thanks Xuan Gong for updating the patch! Latest patch get much closer. + private static final String MAX_LOG_AGGREGATION_DIAGNOSTICS_IN_MEMORY = + YarnConfiguration.RM_PREFIX + "max-log-aggregation-diagnostics-in-memory" ; + private static int DEFAULT_MAX_LOG_AGGREGATION_DIAGNOSTICS_IN_MEMORY = 10; Do we want to expose this configuration to user? If so, we should put it in yarn-default.xml with properly description on it. + // If the log aggregation status got from latest nm heartbeat + // is Running, and current log aggregation status is TimeOut, + // based on whether there are any failure messages for this NM, + // we will reset the log aggregation status as RUNNING or + // RUNNING_WITH_FAILURE + if (logAggregationFailureMessagesForNMs.get(nodeId) != null + && !logAggregationFailureMessagesForNMs.get(nodeId).isEmpty()) { + curReport + .setLogAggregationStatus(LogAggregationStatus.RUNNING_WITH_FAILURE); + } + } + curReport.setLogAggregationStatus(report + .getLogAggregationStatus()); "curReport.setLogAggregationStatus(LogAggregationStatus.RUNNING_WITH_FAILURE); " should be replaced by "report.setLogAggregationStatus(..." ? Or it could be override later by RUNNING. Other looks fine to me.
          Hide
          xgong Xuan Gong added a comment -

          new patch addressed all the latest comments

          Show
          xgong Xuan Gong added a comment - new patch addressed all the latest comments
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 43s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 32s There were no new javac warning messages.
          +1 javadoc 9m 33s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 2m 21s The applied patch generated 1 new checkstyle issues (total was 212, now 213).
          -1 checkstyle 3m 6s The applied patch generated 2 new checkstyle issues (total was 70, now 63).
          +1 whitespace 0m 25s The patch has no lines that end in whitespace.
          +1 install 1m 34s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 5m 54s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 yarn tests 0m 22s Tests passed in hadoop-yarn-api.
          +1 yarn tests 2m 1s Tests passed in hadoop-yarn-common.
          +1 yarn tests 0m 24s Tests passed in hadoop-yarn-server-common.
          +1 yarn tests 5m 58s Tests passed in hadoop-yarn-server-nodemanager.
          +1 yarn tests 52m 12s Tests passed in hadoop-yarn-server-resourcemanager.
              105m 9s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12732363/YARN-3505.5.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / f24452d
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7902/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt https://builds.apache.org/job/PreCommit-YARN-Build/7902/artifact/patchprocess/diffcheckstylehadoop-yarn-server-common.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/7902/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/7902/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-common test log https://builds.apache.org/job/PreCommit-YARN-Build/7902/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt
          hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7902/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7902/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7902/testReport/
          Java 1.7.0_55
          uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/7902/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 43s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 32s There were no new javac warning messages. +1 javadoc 9m 33s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 21s The applied patch generated 1 new checkstyle issues (total was 212, now 213). -1 checkstyle 3m 6s The applied patch generated 2 new checkstyle issues (total was 70, now 63). +1 whitespace 0m 25s The patch has no lines that end in whitespace. +1 install 1m 34s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 5m 54s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 yarn tests 0m 22s Tests passed in hadoop-yarn-api. +1 yarn tests 2m 1s Tests passed in hadoop-yarn-common. +1 yarn tests 0m 24s Tests passed in hadoop-yarn-server-common. +1 yarn tests 5m 58s Tests passed in hadoop-yarn-server-nodemanager. +1 yarn tests 52m 12s Tests passed in hadoop-yarn-server-resourcemanager.     105m 9s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12732363/YARN-3505.5.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / f24452d checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7902/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt https://builds.apache.org/job/PreCommit-YARN-Build/7902/artifact/patchprocess/diffcheckstylehadoop-yarn-server-common.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/7902/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/7902/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-common test log https://builds.apache.org/job/PreCommit-YARN-Build/7902/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7902/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7902/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7902/testReport/ Java 1.7.0_55 uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/7902/console This message was automatically generated.
          Hide
          djp Junping Du added a comment -

          Latest patch LGTM.
          Jian He, about your previous comments, LogAggregationReport#(get/set)getNodeId already been removed, and LogAggregationReport#(get/set)DiagnosticMessage is necessary (see: AppLogAggregatorImpl.java). Any further comments from you?

          Show
          djp Junping Du added a comment - Latest patch LGTM. Jian He , about your previous comments, LogAggregationReport#(get/set)getNodeId already been removed, and LogAggregationReport#(get/set)DiagnosticMessage is necessary (see: AppLogAggregatorImpl.java). Any further comments from you?
          Hide
          djp Junping Du added a comment -

          Fix two checkstyle NITs in .6 patch, still left one (method over 150 lines) which doesn't matter. Will commit the patch if this is the only warning from Jenkins.

          Show
          djp Junping Du added a comment - Fix two checkstyle NITs in .6 patch, still left one (method over 150 lines) which doesn't matter. Will commit the patch if this is the only warning from Jenkins.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 43s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 33s There were no new javac warning messages.
          +1 javadoc 9m 34s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 3m 4s The applied patch generated 1 new checkstyle issues (total was 71, now 63).
          +1 whitespace 0m 25s The patch has no lines that end in whitespace.
          +1 install 1m 34s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 5m 51s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 yarn tests 0m 21s Tests passed in hadoop-yarn-api.
          +1 yarn tests 1m 56s Tests passed in hadoop-yarn-common.
          +1 yarn tests 0m 24s Tests passed in hadoop-yarn-server-common.
          +1 yarn tests 6m 9s Tests passed in hadoop-yarn-server-nodemanager.
          -1 yarn tests 59m 59s Tests failed in hadoop-yarn-server-resourcemanager.
              112m 54s  



          Reason Tests
          Timed out tests org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12732825/YARN-3505.6.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / def9136
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7933/artifact/patchprocess/diffcheckstylehadoop-yarn-server-common.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/7933/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/7933/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-common test log https://builds.apache.org/job/PreCommit-YARN-Build/7933/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt
          hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7933/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7933/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7933/testReport/
          Java 1.7.0_55
          uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/7933/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 43s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 33s There were no new javac warning messages. +1 javadoc 9m 34s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 3m 4s The applied patch generated 1 new checkstyle issues (total was 71, now 63). +1 whitespace 0m 25s The patch has no lines that end in whitespace. +1 install 1m 34s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 5m 51s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 yarn tests 0m 21s Tests passed in hadoop-yarn-api. +1 yarn tests 1m 56s Tests passed in hadoop-yarn-common. +1 yarn tests 0m 24s Tests passed in hadoop-yarn-server-common. +1 yarn tests 6m 9s Tests passed in hadoop-yarn-server-nodemanager. -1 yarn tests 59m 59s Tests failed in hadoop-yarn-server-resourcemanager.     112m 54s   Reason Tests Timed out tests org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12732825/YARN-3505.6.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / def9136 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7933/artifact/patchprocess/diffcheckstylehadoop-yarn-server-common.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/7933/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/7933/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-common test log https://builds.apache.org/job/PreCommit-YARN-Build/7933/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7933/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7933/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7933/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/7933/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 43s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 32s There were no new javac warning messages.
          +1 javadoc 9m 35s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 3m 6s The applied patch generated 1 new checkstyle issues (total was 70, now 62).
          +1 whitespace 0m 24s The patch has no lines that end in whitespace.
          +1 install 1m 34s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 5m 54s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 yarn tests 0m 25s Tests passed in hadoop-yarn-api.
          +1 yarn tests 1m 57s Tests passed in hadoop-yarn-common.
          +1 yarn tests 0m 24s Tests passed in hadoop-yarn-server-common.
          +1 yarn tests 6m 9s Tests passed in hadoop-yarn-server-nodemanager.
          +1 yarn tests 53m 12s Tests passed in hadoop-yarn-server-resourcemanager.
              106m 16s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12732825/YARN-3505.6.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / def9136
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7935/artifact/patchprocess/diffcheckstylehadoop-yarn-server-common.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/7935/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/7935/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-common test log https://builds.apache.org/job/PreCommit-YARN-Build/7935/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt
          hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7935/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7935/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7935/testReport/
          Java 1.7.0_55
          uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/7935/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 43s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 32s There were no new javac warning messages. +1 javadoc 9m 35s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 3m 6s The applied patch generated 1 new checkstyle issues (total was 70, now 62). +1 whitespace 0m 24s The patch has no lines that end in whitespace. +1 install 1m 34s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 5m 54s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 yarn tests 0m 25s Tests passed in hadoop-yarn-api. +1 yarn tests 1m 57s Tests passed in hadoop-yarn-common. +1 yarn tests 0m 24s Tests passed in hadoop-yarn-server-common. +1 yarn tests 6m 9s Tests passed in hadoop-yarn-server-nodemanager. +1 yarn tests 53m 12s Tests passed in hadoop-yarn-server-resourcemanager.     106m 16s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12732825/YARN-3505.6.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / def9136 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7935/artifact/patchprocess/diffcheckstylehadoop-yarn-server-common.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/7935/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/7935/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-common test log https://builds.apache.org/job/PreCommit-YARN-Build/7935/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7935/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7935/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7935/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/7935/console This message was automatically generated.
          Hide
          djp Junping Du added a comment -

          I have commit latest patch to trunk and branch-2. Thanks Xuan Gong for the patch contribution and review comments from Jian He and Vinod Kumar Vavilapalli!

          Show
          djp Junping Du added a comment - I have commit latest patch to trunk and branch-2. Thanks Xuan Gong for the patch contribution and review comments from Jian He and Vinod Kumar Vavilapalli !
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #7833 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7833/)
          YARN-3505. Node's Log Aggregation Report with SUCCEED should not cached in RMApps. Contributed by Xuan Gong. (junping_du: rev 15ccd967ee3e7046a50522089f67ba01f36ec76a)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/LogAggregationReport.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeStatusEvent.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppLogAggregationStatusBlock.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/NodeHeartbeatRequestPBImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/AppBlock.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/logaggregationstatus/TestRMAppLogAggregationStatus.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/LogAggregationStatus.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppBlock.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/LogAggregationReportPBImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/NodeHeartbeatRequest.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/proto/yarn_server_common_service_protos.proto
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #7833 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7833/ ) YARN-3505 . Node's Log Aggregation Report with SUCCEED should not cached in RMApps. Contributed by Xuan Gong. (junping_du: rev 15ccd967ee3e7046a50522089f67ba01f36ec76a) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/LogAggregationReport.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeStatusEvent.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppLogAggregationStatusBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/NodeHeartbeatRequestPBImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/AppBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/logaggregationstatus/TestRMAppLogAggregationStatus.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/LogAggregationStatus.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppBlock.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/LogAggregationReportPBImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/NodeHeartbeatRequest.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/proto/yarn_server_common_service_protos.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          Hide
          djp Junping Du added a comment -

          Forget to mention, given the previous patch YARN-1402 is in 2.8 only and this patch should apply on that one. Mark target/fixed version as 2.8.
          Still in struggle if we should move these two patches to 2.7.1 or better to keep them in 2.8. Vinod Kumar Vavilapalli, any suggestions here?

          Show
          djp Junping Du added a comment - Forget to mention, given the previous patch YARN-1402 is in 2.8 only and this patch should apply on that one. Mark target/fixed version as 2.8. Still in struggle if we should move these two patches to 2.7.1 or better to keep them in 2.8. Vinod Kumar Vavilapalli , any suggestions here?
          Hide
          xgong Xuan Gong added a comment -

          reopen the ticket and attached an addendum patch...

                if (appFinished) {
                  // If the app is finished, one extra final report with log aggregation
                  // status SUCCEEDED/FAILED will be sent to RM to inform the RM
                  // that the log aggregation in this NM is completed.
                  LogAggregationReport finalReport =
                      Records.newRecord(LogAggregationReport.class);
                  finalReport.setApplicationId(appId);
                  finalReport.setLogAggregationStatus(renameTemporaryLogFileFailed
                      ? LogAggregationStatus.FAILED : LogAggregationStatus.SUCCEEDED);
                  this.context.getLogAggregationStatusForApps().add(report);
                }
          

          Should attach the finalReport ...

          this.context.getLogAggregationStatusForApps().add(finalReport);
          
          Show
          xgong Xuan Gong added a comment - reopen the ticket and attached an addendum patch... if (appFinished) { // If the app is finished, one extra final report with log aggregation // status SUCCEEDED/FAILED will be sent to RM to inform the RM // that the log aggregation in this NM is completed. LogAggregationReport finalReport = Records.newRecord(LogAggregationReport.class); finalReport.setApplicationId(appId); finalReport.setLogAggregationStatus(renameTemporaryLogFileFailed ? LogAggregationStatus.FAILED : LogAggregationStatus.SUCCEEDED); this .context.getLogAggregationStatusForApps().add(report); } Should attach the finalReport ... this .context.getLogAggregationStatusForApps().add(finalReport);
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 48s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 7m 36s There were no new javac warning messages.
          +1 javadoc 9m 45s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 0m 40s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 35s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 1m 2s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 yarn tests 5m 59s Tests passed in hadoop-yarn-server-nodemanager.
              42m 25s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12733034/YARN-3505.addendum.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 9a2a955
          hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7946/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7946/testReport/
          Java 1.7.0_55
          uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/7946/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 48s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 36s There were no new javac warning messages. +1 javadoc 9m 45s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 40s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 35s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 2s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 yarn tests 5m 59s Tests passed in hadoop-yarn-server-nodemanager.     42m 25s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12733034/YARN-3505.addendum.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 9a2a955 hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7946/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7946/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/7946/console This message was automatically generated.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #928 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/928/)
          YARN-3505. Node's Log Aggregation Report with SUCCEED should not cached in RMApps. Contributed by Xuan Gong. (junping_du: rev 15ccd967ee3e7046a50522089f67ba01f36ec76a)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/LogAggregationReport.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppLogAggregationStatusBlock.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/AppBlock.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeStatusEvent.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/NodeHeartbeatRequest.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/logaggregationstatus/TestRMAppLogAggregationStatus.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppBlock.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/NodeHeartbeatRequestPBImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/LogAggregationStatus.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/LogAggregationReportPBImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/proto/yarn_server_common_service_protos.proto
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #928 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/928/ ) YARN-3505 . Node's Log Aggregation Report with SUCCEED should not cached in RMApps. Contributed by Xuan Gong. (junping_du: rev 15ccd967ee3e7046a50522089f67ba01f36ec76a) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/LogAggregationReport.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppLogAggregationStatusBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/AppBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeStatusEvent.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/NodeHeartbeatRequest.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/logaggregationstatus/TestRMAppLogAggregationStatus.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/NodeHeartbeatRequestPBImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/LogAggregationStatus.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/LogAggregationReportPBImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/proto/yarn_server_common_service_protos.proto
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #197 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/197/)
          YARN-3505. Node's Log Aggregation Report with SUCCEED should not cached in RMApps. Contributed by Xuan Gong. (junping_du: rev 15ccd967ee3e7046a50522089f67ba01f36ec76a)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/logaggregationstatus/TestRMAppLogAggregationStatus.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppLogAggregationStatusBlock.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/LogAggregationReport.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeStatusEvent.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/LogAggregationStatus.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/NodeHeartbeatRequestPBImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/LogAggregationReportPBImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/NodeHeartbeatRequest.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/proto/yarn_server_common_service_protos.proto
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppBlock.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/AppBlock.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #197 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/197/ ) YARN-3505 . Node's Log Aggregation Report with SUCCEED should not cached in RMApps. Contributed by Xuan Gong. (junping_du: rev 15ccd967ee3e7046a50522089f67ba01f36ec76a) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/logaggregationstatus/TestRMAppLogAggregationStatus.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppLogAggregationStatusBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/LogAggregationReport.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeStatusEvent.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/LogAggregationStatus.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/NodeHeartbeatRequestPBImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/LogAggregationReportPBImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/NodeHeartbeatRequest.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/proto/yarn_server_common_service_protos.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/AppBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          Hide
          djp Junping Du added a comment -

          +1. Commit the addendum patch to trunk and branch-2.

          Show
          djp Junping Du added a comment - +1. Commit the addendum patch to trunk and branch-2.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #7841 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7841/)
          YARN-3505 addendum: fix an issue in previous patch. (junping_du: rev 03a293aed6de101b0cae1a294f506903addcaa75)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7841 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7841/ ) YARN-3505 addendum: fix an issue in previous patch. (junping_du: rev 03a293aed6de101b0cae1a294f506903addcaa75) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2126 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2126/)
          YARN-3505. Node's Log Aggregation Report with SUCCEED should not cached in RMApps. Contributed by Xuan Gong. (junping_du: rev 15ccd967ee3e7046a50522089f67ba01f36ec76a)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/LogAggregationStatus.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/LogAggregationReportPBImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/proto/yarn_server_common_service_protos.proto
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/LogAggregationReport.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/NodeHeartbeatRequestPBImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/logaggregationstatus/TestRMAppLogAggregationStatus.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppBlock.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/AppBlock.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppLogAggregationStatusBlock.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/NodeHeartbeatRequest.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeStatusEvent.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2126 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2126/ ) YARN-3505 . Node's Log Aggregation Report with SUCCEED should not cached in RMApps. Contributed by Xuan Gong. (junping_du: rev 15ccd967ee3e7046a50522089f67ba01f36ec76a) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/LogAggregationStatus.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/LogAggregationReportPBImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/proto/yarn_server_common_service_protos.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/LogAggregationReport.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/NodeHeartbeatRequestPBImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/logaggregationstatus/TestRMAppLogAggregationStatus.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/AppBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppLogAggregationStatusBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/NodeHeartbeatRequest.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeStatusEvent.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #186 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/186/)
          YARN-3505. Node's Log Aggregation Report with SUCCEED should not cached in RMApps. Contributed by Xuan Gong. (junping_du: rev 15ccd967ee3e7046a50522089f67ba01f36ec76a)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/LogAggregationReport.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/NodeHeartbeatRequestPBImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/proto/yarn_server_common_service_protos.proto
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppLogAggregationStatusBlock.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/AppBlock.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeStatusEvent.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/logaggregationstatus/TestRMAppLogAggregationStatus.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/NodeHeartbeatRequest.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/LogAggregationReportPBImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/LogAggregationStatus.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppBlock.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #186 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/186/ ) YARN-3505 . Node's Log Aggregation Report with SUCCEED should not cached in RMApps. Contributed by Xuan Gong. (junping_du: rev 15ccd967ee3e7046a50522089f67ba01f36ec76a) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/LogAggregationReport.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/NodeHeartbeatRequestPBImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/proto/yarn_server_common_service_protos.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppLogAggregationStatusBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/AppBlock.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeStatusEvent.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/logaggregationstatus/TestRMAppLogAggregationStatus.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/NodeHeartbeatRequest.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/LogAggregationReportPBImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/LogAggregationStatus.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #196 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/196/)
          YARN-3505. Node's Log Aggregation Report with SUCCEED should not cached in RMApps. Contributed by Xuan Gong. (junping_du: rev 15ccd967ee3e7046a50522089f67ba01f36ec76a)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/logaggregationstatus/TestRMAppLogAggregationStatus.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppLogAggregationStatusBlock.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/LogAggregationReportPBImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/LogAggregationStatus.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/LogAggregationReport.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/NodeHeartbeatRequest.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/AppBlock.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeStatusEvent.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppBlock.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/NodeHeartbeatRequestPBImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/proto/yarn_server_common_service_protos.proto
            YARN-3505 addendum: fix an issue in previous patch. (junping_du: rev 03a293aed6de101b0cae1a294f506903addcaa75)
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #196 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/196/ ) YARN-3505 . Node's Log Aggregation Report with SUCCEED should not cached in RMApps. Contributed by Xuan Gong. (junping_du: rev 15ccd967ee3e7046a50522089f67ba01f36ec76a) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/logaggregationstatus/TestRMAppLogAggregationStatus.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppLogAggregationStatusBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/LogAggregationReportPBImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/LogAggregationStatus.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/LogAggregationReport.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/NodeHeartbeatRequest.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/AppBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeStatusEvent.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/NodeHeartbeatRequestPBImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/proto/yarn_server_common_service_protos.proto YARN-3505 addendum: fix an issue in previous patch. (junping_du: rev 03a293aed6de101b0cae1a294f506903addcaa75) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2144 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2144/)
          YARN-3505. Node's Log Aggregation Report with SUCCEED should not cached in RMApps. Contributed by Xuan Gong. (junping_du: rev 15ccd967ee3e7046a50522089f67ba01f36ec76a)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/AppBlock.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/proto/yarn_server_common_service_protos.proto
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/LogAggregationReportPBImpl.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppBlock.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/LogAggregationStatus.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeStatusEvent.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/logaggregationstatus/TestRMAppLogAggregationStatus.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/NodeHeartbeatRequestPBImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppLogAggregationStatusBlock.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/LogAggregationReport.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/NodeHeartbeatRequest.java
            YARN-3505 addendum: fix an issue in previous patch. (junping_du: rev 03a293aed6de101b0cae1a294f506903addcaa75)
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2144 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2144/ ) YARN-3505 . Node's Log Aggregation Report with SUCCEED should not cached in RMApps. Contributed by Xuan Gong. (junping_du: rev 15ccd967ee3e7046a50522089f67ba01f36ec76a) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/AppBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/proto/yarn_server_common_service_protos.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/LogAggregationReportPBImpl.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/LogAggregationStatus.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeStatusEvent.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/logaggregationstatus/TestRMAppLogAggregationStatus.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/NodeHeartbeatRequestPBImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppLogAggregationStatusBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/LogAggregationReport.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/NodeHeartbeatRequest.java YARN-3505 addendum: fix an issue in previous patch. (junping_du: rev 03a293aed6de101b0cae1a294f506903addcaa75) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #198 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/198/)
          YARN-3505 addendum: fix an issue in previous patch. (junping_du: rev 03a293aed6de101b0cae1a294f506903addcaa75)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #198 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/198/ ) YARN-3505 addendum: fix an issue in previous patch. (junping_du: rev 03a293aed6de101b0cae1a294f506903addcaa75) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #929 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/929/)
          YARN-3505 addendum: fix an issue in previous patch. (junping_du: rev 03a293aed6de101b0cae1a294f506903addcaa75)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #929 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/929/ ) YARN-3505 addendum: fix an issue in previous patch. (junping_du: rev 03a293aed6de101b0cae1a294f506903addcaa75) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2127 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2127/)
          YARN-3505 addendum: fix an issue in previous patch. (junping_du: rev 03a293aed6de101b0cae1a294f506903addcaa75)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2127 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2127/ ) YARN-3505 addendum: fix an issue in previous patch. (junping_du: rev 03a293aed6de101b0cae1a294f506903addcaa75) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #187 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/187/)
          YARN-3505 addendum: fix an issue in previous patch. (junping_du: rev 03a293aed6de101b0cae1a294f506903addcaa75)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #187 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/187/ ) YARN-3505 addendum: fix an issue in previous patch. (junping_du: rev 03a293aed6de101b0cae1a294f506903addcaa75) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java
          Hide
          zjshen Zhijie Shen added a comment - - edited

          Are the followings the incompatible change: deleting the field and changing the type? /cc Xuan Gong

          @@ -50,19 +50,13 @@ message NodeHeartbeatRequestProto {
             optional MasterKeyProto last_known_container_token_master_key = 2;
             optional MasterKeyProto last_known_nm_token_master_key = 3;
             optional StringArrayProto nodeLabels = 4;
          -  repeated LogAggregationReportsForAppsProto log_aggregation_reports_for_apps = 5;
          -}
          
           message LogAggregationReportProto {
          -optional ApplicationIdProto application_id = 1;
          -optional NodeIdProto node_id = 2;
          -optional LogAggregationStatusProto log_aggregation_status = 3;
          -optional string diagnostics = 4 [default = "N/A"];
          +  optional ApplicationIdProto application_id = 1;
          +  optional LogAggregationStatusProto log_aggregation_status = 2;
          +  optional string diagnostics = 3 [default = "N/A"];
           }
          
          Show
          zjshen Zhijie Shen added a comment - - edited Are the followings the incompatible change: deleting the field and changing the type? /cc Xuan Gong @@ -50,19 +50,13 @@ message NodeHeartbeatRequestProto { optional MasterKeyProto last_known_container_token_master_key = 2; optional MasterKeyProto last_known_nm_token_master_key = 3; optional StringArrayProto nodeLabels = 4; - repeated LogAggregationReportsForAppsProto log_aggregation_reports_for_apps = 5; -} message LogAggregationReportProto { -optional ApplicationIdProto application_id = 1; -optional NodeIdProto node_id = 2; -optional LogAggregationStatusProto log_aggregation_status = 3; -optional string diagnostics = 4 [ default = "N/A" ]; + optional ApplicationIdProto application_id = 1; + optional LogAggregationStatusProto log_aggregation_status = 2; + optional string diagnostics = 3 [ default = "N/A" ]; }
          Hide
          xgong Xuan Gong added a comment -

          Zhijie Shen Do not think so. Both of the changes happen in 2.8. So, should be fine.

          Show
          xgong Xuan Gong added a comment - Zhijie Shen Do not think so. Both of the changes happen in 2.8. So, should be fine.

            People

            • Assignee:
              xgong Xuan Gong
              Reporter:
              djp Junping Du
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development