Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4766

NM should not aggregate logs older than the retention policy

    Details

    • Hadoop Flags:
      Reviewed

      Description

      When a log aggregation fails on the NM the information is for the attempt is kept in the recovery DB. Log aggregation can fail for multiple reasons which are often related to HDFS space or permissions.

      On restart the recovery DB is read and if an application attempt needs its logs aggregated, the files are scheduled for aggregation without any checks. The log files could be older than the retention limit in which case we should not aggregate them but immediately mark them for deletion from the local file system.

      1. yarn4766.001.patch
        54 kB
        Haibo Chen
      2. yarn4766.002.patch
        54 kB
        Haibo Chen
      3. yarn4766.003.patch
        55 kB
        Haibo Chen
      4. yarn4766.004.patch
        53 kB
        Haibo Chen
      5. yarn4766.005.patch
        52 kB
        Haibo Chen
      6. yarn4766.006.patch
        51 kB
        Haibo Chen

        Activity

        Hide
        haibochen Haibo Chen added a comment -

        Use APPLICATION_LOG_HANDLING_INITED timestamp as the starting point of application log aggregation, which is saved in NMStateStore. If the time elapsed since APPLICATION_LOG_HANDLING_INITED is more than the configured retention policy, application log files will not be aggregated but scheduled for deletion.

        Show
        haibochen Haibo Chen added a comment - Use APPLICATION_LOG_HANDLING_INITED timestamp as the starting point of application log aggregation, which is saved in NMStateStore. If the time elapsed since APPLICATION_LOG_HANDLING_INITED is more than the configured retention policy, application log files will not be aggregated but scheduled for deletion.
        Hide
        haibochen Haibo Chen added a comment -

        Use APPLICATION_LOG_HANDLING_INITED timestamp as the starting point of application log aggregation, which is saved in NMStateStore. If the time elapsed since APPLICATION_LOG_HANDLING_INITED is more than the configured retention policy, application log files will not be aggregated but scheduled for deletion.

        Show
        haibochen Haibo Chen added a comment - Use APPLICATION_LOG_HANDLING_INITED timestamp as the starting point of application log aggregation, which is saved in NMStateStore. If the time elapsed since APPLICATION_LOG_HANDLING_INITED is more than the configured retention policy, application log files will not be aggregated but scheduled for deletion.
        Hide
        haibochen Haibo Chen added a comment -

        Use APPLICATION_LOG_HANDLING_INITED timestamp as the starting point of application log aggregation, which is saved in NMStateStore. If the time elapsed since APPLICATION_LOG_HANDLING_INITED is more than the configured retention policy, application log files will not be aggregated but scheduled for deletion.

        Show
        haibochen Haibo Chen added a comment - Use APPLICATION_LOG_HANDLING_INITED timestamp as the starting point of application log aggregation, which is saved in NMStateStore. If the time elapsed since APPLICATION_LOG_HANDLING_INITED is more than the configured retention policy, application log files will not be aggregated but scheduled for deletion.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 11s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
        0 mvndep 0m 33s Maven dependency ordering for branch
        +1 mvninstall 6m 46s trunk passed
        +1 compile 1m 51s trunk passed with JDK v1.8.0_74
        +1 compile 2m 7s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 34s trunk passed
        +1 mvnsite 1m 0s trunk passed
        +1 mvneclipse 0m 26s trunk passed
        +1 findbugs 2m 0s trunk passed
        +1 javadoc 0m 45s trunk passed with JDK v1.8.0_74
        +1 javadoc 0m 56s trunk passed with JDK v1.7.0_95
        0 mvndep 0m 10s Maven dependency ordering for patch
        +1 mvninstall 0m 50s the patch passed
        +1 compile 1m 46s the patch passed with JDK v1.8.0_74
        +1 cc 1m 46s the patch passed
        +1 javac 1m 46s the patch passed
        +1 compile 2m 3s the patch passed with JDK v1.7.0_95
        +1 cc 2m 3s the patch passed
        +1 javac 2m 3s the patch passed
        -1 checkstyle 0m 33s hadoop-yarn-project/hadoop-yarn: patch generated 11 new + 123 unchanged - 2 fixed = 134 total (was 125)
        +1 mvnsite 0m 54s the patch passed
        +1 mvneclipse 0m 23s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 22s the patch passed
        +1 javadoc 0m 42s the patch passed with JDK v1.8.0_74
        +1 javadoc 0m 52s the patch passed with JDK v1.7.0_95
        +1 unit 1m 53s hadoop-yarn-common in the patch passed with JDK v1.8.0_74.
        +1 unit 9m 5s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_74.
        +1 unit 2m 10s hadoop-yarn-common in the patch passed with JDK v1.7.0_95.
        +1 unit 9m 36s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_95.
        -1 asflicense 0m 19s Patch generated 1 ASF License warnings.
        52m 1s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12793383/yarn4766.001.patch
        JIRA Issue YARN-4766
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux c571d8bff901 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / bd5556b
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10778/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10778/testReport/
        asflicense https://builds.apache.org/job/PreCommit-YARN-Build/10778/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/10778/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 33s Maven dependency ordering for branch +1 mvninstall 6m 46s trunk passed +1 compile 1m 51s trunk passed with JDK v1.8.0_74 +1 compile 2m 7s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 34s trunk passed +1 mvnsite 1m 0s trunk passed +1 mvneclipse 0m 26s trunk passed +1 findbugs 2m 0s trunk passed +1 javadoc 0m 45s trunk passed with JDK v1.8.0_74 +1 javadoc 0m 56s trunk passed with JDK v1.7.0_95 0 mvndep 0m 10s Maven dependency ordering for patch +1 mvninstall 0m 50s the patch passed +1 compile 1m 46s the patch passed with JDK v1.8.0_74 +1 cc 1m 46s the patch passed +1 javac 1m 46s the patch passed +1 compile 2m 3s the patch passed with JDK v1.7.0_95 +1 cc 2m 3s the patch passed +1 javac 2m 3s the patch passed -1 checkstyle 0m 33s hadoop-yarn-project/hadoop-yarn: patch generated 11 new + 123 unchanged - 2 fixed = 134 total (was 125) +1 mvnsite 0m 54s the patch passed +1 mvneclipse 0m 23s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 22s the patch passed +1 javadoc 0m 42s the patch passed with JDK v1.8.0_74 +1 javadoc 0m 52s the patch passed with JDK v1.7.0_95 +1 unit 1m 53s hadoop-yarn-common in the patch passed with JDK v1.8.0_74. +1 unit 9m 5s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_74. +1 unit 2m 10s hadoop-yarn-common in the patch passed with JDK v1.7.0_95. +1 unit 9m 36s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_95. -1 asflicense 0m 19s Patch generated 1 ASF License warnings. 52m 1s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12793383/yarn4766.001.patch JIRA Issue YARN-4766 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux c571d8bff901 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / bd5556b Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10778/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10778/testReport/ asflicense https://builds.apache.org/job/PreCommit-YARN-Build/10778/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/10778/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        haibochen Haibo Chen added a comment -

        Addressed checkstyle issues

        Show
        haibochen Haibo Chen added a comment - Addressed checkstyle issues
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
        0 mvndep 0m 34s Maven dependency ordering for branch
        +1 mvninstall 9m 42s trunk passed
        +1 compile 3m 59s trunk passed with JDK v1.8.0_74
        +1 compile 2m 56s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 41s trunk passed
        +1 mvnsite 1m 13s trunk passed
        +1 mvneclipse 0m 29s trunk passed
        +1 findbugs 2m 30s trunk passed
        +1 javadoc 1m 15s trunk passed with JDK v1.8.0_74
        +1 javadoc 1m 5s trunk passed with JDK v1.7.0_95
        0 mvndep 0m 11s Maven dependency ordering for patch
        +1 mvninstall 1m 5s the patch passed
        +1 compile 3m 32s the patch passed with JDK v1.8.0_74
        +1 cc 3m 32s the patch passed
        +1 javac 3m 32s the patch passed
        +1 compile 2m 54s the patch passed with JDK v1.7.0_95
        +1 cc 2m 54s the patch passed
        +1 javac 2m 54s the patch passed
        -1 checkstyle 0m 37s hadoop-yarn-project/hadoop-yarn: patch generated 2 new + 121 unchanged - 2 fixed = 123 total (was 123)
        +1 mvnsite 1m 11s the patch passed
        +1 mvneclipse 0m 26s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 3m 1s the patch passed
        +1 javadoc 1m 8s the patch passed with JDK v1.8.0_74
        +1 javadoc 1m 3s the patch passed with JDK v1.7.0_95
        +1 unit 2m 54s hadoop-yarn-common in the patch passed with JDK v1.8.0_74.
        -1 unit 11m 34s hadoop-yarn-server-nodemanager in the patch failed with JDK v1.8.0_74.
        +1 unit 2m 33s hadoop-yarn-common in the patch passed with JDK v1.7.0_95.
        +1 unit 10m 36s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_95.
        -1 asflicense 0m 24s Patch generated 1 ASF License warnings.
        69m 19s



        Reason Tests
        JDK v1.8.0_74 Failed junit tests hadoop.yarn.server.nodemanager.containermanager.logaggregation.TestLogAggregationService



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12793443/yarn4766.002.patch
        JIRA Issue YARN-4766
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux 054bc07bbbfb 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 19e8f07
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10783/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/10783/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdk1.8.0_74.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10783/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdk1.8.0_74.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10783/testReport/
        asflicense https://builds.apache.org/job/PreCommit-YARN-Build/10783/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/10783/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 34s Maven dependency ordering for branch +1 mvninstall 9m 42s trunk passed +1 compile 3m 59s trunk passed with JDK v1.8.0_74 +1 compile 2m 56s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 41s trunk passed +1 mvnsite 1m 13s trunk passed +1 mvneclipse 0m 29s trunk passed +1 findbugs 2m 30s trunk passed +1 javadoc 1m 15s trunk passed with JDK v1.8.0_74 +1 javadoc 1m 5s trunk passed with JDK v1.7.0_95 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 1m 5s the patch passed +1 compile 3m 32s the patch passed with JDK v1.8.0_74 +1 cc 3m 32s the patch passed +1 javac 3m 32s the patch passed +1 compile 2m 54s the patch passed with JDK v1.7.0_95 +1 cc 2m 54s the patch passed +1 javac 2m 54s the patch passed -1 checkstyle 0m 37s hadoop-yarn-project/hadoop-yarn: patch generated 2 new + 121 unchanged - 2 fixed = 123 total (was 123) +1 mvnsite 1m 11s the patch passed +1 mvneclipse 0m 26s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 3m 1s the patch passed +1 javadoc 1m 8s the patch passed with JDK v1.8.0_74 +1 javadoc 1m 3s the patch passed with JDK v1.7.0_95 +1 unit 2m 54s hadoop-yarn-common in the patch passed with JDK v1.8.0_74. -1 unit 11m 34s hadoop-yarn-server-nodemanager in the patch failed with JDK v1.8.0_74. +1 unit 2m 33s hadoop-yarn-common in the patch passed with JDK v1.7.0_95. +1 unit 10m 36s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_95. -1 asflicense 0m 24s Patch generated 1 ASF License warnings. 69m 19s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.yarn.server.nodemanager.containermanager.logaggregation.TestLogAggregationService Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12793443/yarn4766.002.patch JIRA Issue YARN-4766 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 054bc07bbbfb 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 19e8f07 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10783/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10783/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdk1.8.0_74.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10783/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdk1.8.0_74.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10783/testReport/ asflicense https://builds.apache.org/job/PreCommit-YARN-Build/10783/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/10783/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        haibochen Haibo Chen added a comment -

        fixed the liscense + checkstyle issues. The unit test failure is unrelated to the patch. I have create another jira to fix the test failure at https://issues.apache.org/jira/browse/YARN-4838

        Show
        haibochen Haibo Chen added a comment - fixed the liscense + checkstyle issues. The unit test failure is unrelated to the patch. I have create another jira to fix the test failure at https://issues.apache.org/jira/browse/YARN-4838
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 11s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
        0 mvndep 0m 20s Maven dependency ordering for branch
        +1 mvninstall 6m 36s trunk passed
        +1 compile 1m 50s trunk passed with JDK v1.8.0_74
        +1 compile 2m 5s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 35s trunk passed
        +1 mvnsite 0m 57s trunk passed
        +1 mvneclipse 0m 24s trunk passed
        +1 findbugs 1m 58s trunk passed
        +1 javadoc 0m 45s trunk passed with JDK v1.8.0_74
        +1 javadoc 0m 55s trunk passed with JDK v1.7.0_95
        0 mvndep 0m 10s Maven dependency ordering for patch
        +1 mvninstall 0m 49s the patch passed
        +1 compile 1m 49s the patch passed with JDK v1.8.0_74
        +1 cc 1m 49s the patch passed
        +1 javac 1m 49s the patch passed
        +1 compile 2m 2s the patch passed with JDK v1.7.0_95
        +1 cc 2m 2s the patch passed
        +1 javac 2m 2s the patch passed
        -1 checkstyle 0m 32s hadoop-yarn-project/hadoop-yarn: patch generated 1 new + 123 unchanged - 2 fixed = 124 total (was 125)
        +1 mvnsite 0m 53s the patch passed
        +1 mvneclipse 0m 23s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 19s the patch passed
        +1 javadoc 0m 44s the patch passed with JDK v1.8.0_74
        +1 javadoc 0m 51s the patch passed with JDK v1.7.0_95
        +1 unit 1m 54s hadoop-yarn-common in the patch passed with JDK v1.8.0_74.
        +1 unit 9m 3s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_74.
        +1 unit 2m 9s hadoop-yarn-common in the patch passed with JDK v1.7.0_95.
        +1 unit 9m 35s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_95.
        +1 asflicense 0m 19s Patch does not generate ASF License warnings.
        51m 22s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12794094/yarn4766.003.patch
        JIRA Issue YARN-4766
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux d6a9ed546f74 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / dc951e6
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10815/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10815/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/10815/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 20s Maven dependency ordering for branch +1 mvninstall 6m 36s trunk passed +1 compile 1m 50s trunk passed with JDK v1.8.0_74 +1 compile 2m 5s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 35s trunk passed +1 mvnsite 0m 57s trunk passed +1 mvneclipse 0m 24s trunk passed +1 findbugs 1m 58s trunk passed +1 javadoc 0m 45s trunk passed with JDK v1.8.0_74 +1 javadoc 0m 55s trunk passed with JDK v1.7.0_95 0 mvndep 0m 10s Maven dependency ordering for patch +1 mvninstall 0m 49s the patch passed +1 compile 1m 49s the patch passed with JDK v1.8.0_74 +1 cc 1m 49s the patch passed +1 javac 1m 49s the patch passed +1 compile 2m 2s the patch passed with JDK v1.7.0_95 +1 cc 2m 2s the patch passed +1 javac 2m 2s the patch passed -1 checkstyle 0m 32s hadoop-yarn-project/hadoop-yarn: patch generated 1 new + 123 unchanged - 2 fixed = 124 total (was 125) +1 mvnsite 0m 53s the patch passed +1 mvneclipse 0m 23s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 19s the patch passed +1 javadoc 0m 44s the patch passed with JDK v1.8.0_74 +1 javadoc 0m 51s the patch passed with JDK v1.7.0_95 +1 unit 1m 54s hadoop-yarn-common in the patch passed with JDK v1.8.0_74. +1 unit 9m 3s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_74. +1 unit 2m 9s hadoop-yarn-common in the patch passed with JDK v1.7.0_95. +1 unit 9m 35s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 19s Patch does not generate ASF License warnings. 51m 22s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12794094/yarn4766.003.patch JIRA Issue YARN-4766 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux d6a9ed546f74 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / dc951e6 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10815/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10815/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/10815/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        rkanter Robert Kanter added a comment -

        Looks good overall. Here's a few things:

        1. AggregatedLogFormat#getPendingLogFilesToUploadForThisContainer should be marked @VisibleForTesting
        2. It's typically easier to maintain multiple constructors when you have the one with the most arguments do the "real" work and the others just call others with default values in arguments. Can you make ApplicationImpl do that?
        3. In ApplicationImpl#buildAppProto, it's catching an IOException (which shouldn't occur) and simply logging it. If this does somehow occur, then it's going to continue executing, which is probably bad. Given that the only caller of this method is already doing it from a try-catch block, I think we'd be better off throwing the IOException. Also, the caller should log the Exception so that we get a stack trace.
        4. There's an extra newline above ApplicationImpl#ApplicationImpl
        5. AppLogAggregatorImpl#uploadLogsForContainers creates a new array named paths that's never used.
        6. In TestAppLogAggregatorImpl, testAggregatorWithRetentionPolicyDisabled_shouldUploadAllFiles and testAggregatorWhenNoFileOlderThanRetentionPolicy_ShouldUploadAll are identical other than a config property or two. Can we make a helper than has most of the code and pass it the config properties so we can combine the code here?
        7. There's an extra newline in the DeletionServiceDeleteTaskProto proto message
        Show
        rkanter Robert Kanter added a comment - Looks good overall. Here's a few things: AggregatedLogFormat#getPendingLogFilesToUploadForThisContainer should be marked @VisibleForTesting It's typically easier to maintain multiple constructors when you have the one with the most arguments do the "real" work and the others just call others with default values in arguments. Can you make ApplicationImpl do that? In ApplicationImpl#buildAppProto , it's catching an IOException (which shouldn't occur) and simply logging it. If this does somehow occur, then it's going to continue executing, which is probably bad. Given that the only caller of this method is already doing it from a try-catch block, I think we'd be better off throwing the IOException . Also, the caller should log the Exception so that we get a stack trace. There's an extra newline above ApplicationImpl#ApplicationImpl AppLogAggregatorImpl#uploadLogsForContainers creates a new array named paths that's never used. In TestAppLogAggregatorImpl , testAggregatorWithRetentionPolicyDisabled_shouldUploadAllFiles and testAggregatorWhenNoFileOlderThanRetentionPolicy_ShouldUploadAll are identical other than a config property or two. Can we make a helper than has most of the code and pass it the config properties so we can combine the code here? There's an extra newline in the DeletionServiceDeleteTaskProto proto message
        Hide
        haibochen Haibo Chen added a comment -

        @Robert Kanter, thanks very much for your review. I have addressed all issues in the latest patch. For #6, I didn't follow exactly your comments. Instead, a new method that takes configs and expected files. testAggregatorWithRetentionPolicyDisabled_shouldUploadAllFiles and testAggregatorWhenNoFileOlderThanRetentionPolicy_ShouldUploadAll are still very much alike, but most of the code duplication is removed.

        Show
        haibochen Haibo Chen added a comment - @Robert Kanter, thanks very much for your review. I have addressed all issues in the latest patch. For #6, I didn't follow exactly your comments. Instead, a new method that takes configs and expected files. testAggregatorWithRetentionPolicyDisabled_shouldUploadAllFiles and testAggregatorWhenNoFileOlderThanRetentionPolicy_ShouldUploadAll are still very much alike, but most of the code duplication is removed.
        Hide
        rkanter Robert Kanter added a comment -

        Thanks Haibo. For #6, that's what I was getting at.

        +1 pending Jenkins (which I just kicked off because it seems to have ignored us).

        Show
        rkanter Robert Kanter added a comment - Thanks Haibo. For #6, that's what I was getting at. +1 pending Jenkins (which I just kicked off because it seems to have ignored us).
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 9s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
        0 mvndep 0m 10s Maven dependency ordering for branch
        +1 mvninstall 6m 38s trunk passed
        +1 compile 1m 46s trunk passed with JDK v1.8.0_92
        +1 compile 2m 4s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 35s trunk passed
        +1 mvnsite 0m 57s trunk passed
        +1 mvneclipse 0m 25s trunk passed
        +1 findbugs 1m 55s trunk passed
        +1 javadoc 0m 44s trunk passed with JDK v1.8.0_92
        +1 javadoc 0m 53s trunk passed with JDK v1.7.0_95
        0 mvndep 0m 10s Maven dependency ordering for patch
        -1 mvninstall 0m 19s hadoop-yarn-server-nodemanager in the patch failed.
        -1 compile 0m 49s hadoop-yarn in the patch failed with JDK v1.8.0_92.
        -1 cc 0m 49s hadoop-yarn in the patch failed with JDK v1.8.0_92.
        -1 javac 0m 49s hadoop-yarn in the patch failed with JDK v1.8.0_92.
        -1 compile 0m 58s hadoop-yarn in the patch failed with JDK v1.7.0_95.
        -1 cc 0m 58s hadoop-yarn in the patch failed with JDK v1.7.0_95.
        -1 javac 0m 58s hadoop-yarn in the patch failed with JDK v1.7.0_95.
        -1 checkstyle 0m 31s hadoop-yarn-project/hadoop-yarn: patch generated 2 new + 121 unchanged - 4 fixed = 123 total (was 125)
        -1 mvnsite 0m 21s hadoop-yarn-server-nodemanager in the patch failed.
        +1 mvneclipse 0m 20s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        -1 findbugs 0m 14s hadoop-yarn-server-nodemanager in the patch failed.
        +1 javadoc 0m 38s the patch passed with JDK v1.8.0_92
        +1 javadoc 0m 49s the patch passed with JDK v1.7.0_95
        +1 unit 1m 53s hadoop-yarn-common in the patch passed with JDK v1.8.0_92.
        -1 unit 0m 23s hadoop-yarn-server-nodemanager in the patch failed with JDK v1.8.0_92.
        +1 unit 2m 9s hadoop-yarn-common in the patch passed with JDK v1.7.0_95.
        -1 unit 0m 23s hadoop-yarn-server-nodemanager in the patch failed with JDK v1.7.0_95.
        +1 asflicense 0m 17s Patch does not generate ASF License warnings.
        29m 45s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:fbe3e86
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12800300/yarn4766.004.patch
        JIRA Issue YARN-4766
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux eded36a59718 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 6be22dd
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_92 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/11234/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
        compile https://builds.apache.org/job/PreCommit-YARN-Build/11234/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.8.0_92.txt
        cc https://builds.apache.org/job/PreCommit-YARN-Build/11234/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.8.0_92.txt
        javac https://builds.apache.org/job/PreCommit-YARN-Build/11234/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.8.0_92.txt
        compile https://builds.apache.org/job/PreCommit-YARN-Build/11234/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.7.0_95.txt
        cc https://builds.apache.org/job/PreCommit-YARN-Build/11234/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.7.0_95.txt
        javac https://builds.apache.org/job/PreCommit-YARN-Build/11234/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.7.0_95.txt
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11234/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
        mvnsite https://builds.apache.org/job/PreCommit-YARN-Build/11234/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
        findbugs https://builds.apache.org/job/PreCommit-YARN-Build/11234/artifact/patchprocess/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/11234/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdk1.8.0_92.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/11234/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11234/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/11234/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 9s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 10s Maven dependency ordering for branch +1 mvninstall 6m 38s trunk passed +1 compile 1m 46s trunk passed with JDK v1.8.0_92 +1 compile 2m 4s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 35s trunk passed +1 mvnsite 0m 57s trunk passed +1 mvneclipse 0m 25s trunk passed +1 findbugs 1m 55s trunk passed +1 javadoc 0m 44s trunk passed with JDK v1.8.0_92 +1 javadoc 0m 53s trunk passed with JDK v1.7.0_95 0 mvndep 0m 10s Maven dependency ordering for patch -1 mvninstall 0m 19s hadoop-yarn-server-nodemanager in the patch failed. -1 compile 0m 49s hadoop-yarn in the patch failed with JDK v1.8.0_92. -1 cc 0m 49s hadoop-yarn in the patch failed with JDK v1.8.0_92. -1 javac 0m 49s hadoop-yarn in the patch failed with JDK v1.8.0_92. -1 compile 0m 58s hadoop-yarn in the patch failed with JDK v1.7.0_95. -1 cc 0m 58s hadoop-yarn in the patch failed with JDK v1.7.0_95. -1 javac 0m 58s hadoop-yarn in the patch failed with JDK v1.7.0_95. -1 checkstyle 0m 31s hadoop-yarn-project/hadoop-yarn: patch generated 2 new + 121 unchanged - 4 fixed = 123 total (was 125) -1 mvnsite 0m 21s hadoop-yarn-server-nodemanager in the patch failed. +1 mvneclipse 0m 20s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. -1 findbugs 0m 14s hadoop-yarn-server-nodemanager in the patch failed. +1 javadoc 0m 38s the patch passed with JDK v1.8.0_92 +1 javadoc 0m 49s the patch passed with JDK v1.7.0_95 +1 unit 1m 53s hadoop-yarn-common in the patch passed with JDK v1.8.0_92. -1 unit 0m 23s hadoop-yarn-server-nodemanager in the patch failed with JDK v1.8.0_92. +1 unit 2m 9s hadoop-yarn-common in the patch passed with JDK v1.7.0_95. -1 unit 0m 23s hadoop-yarn-server-nodemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 17s Patch does not generate ASF License warnings. 29m 45s Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12800300/yarn4766.004.patch JIRA Issue YARN-4766 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux eded36a59718 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6be22dd Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_92 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/11234/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt compile https://builds.apache.org/job/PreCommit-YARN-Build/11234/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.8.0_92.txt cc https://builds.apache.org/job/PreCommit-YARN-Build/11234/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.8.0_92.txt javac https://builds.apache.org/job/PreCommit-YARN-Build/11234/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.8.0_92.txt compile https://builds.apache.org/job/PreCommit-YARN-Build/11234/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.7.0_95.txt cc https://builds.apache.org/job/PreCommit-YARN-Build/11234/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.7.0_95.txt javac https://builds.apache.org/job/PreCommit-YARN-Build/11234/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.7.0_95.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11234/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt mvnsite https://builds.apache.org/job/PreCommit-YARN-Build/11234/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt findbugs https://builds.apache.org/job/PreCommit-YARN-Build/11234/artifact/patchprocess/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11234/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdk1.8.0_92.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11234/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11234/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/11234/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        haibochen Haibo Chen added a comment -

        rebased the patch on the latest trunk, fixed the compilation issue

        Show
        haibochen Haibo Chen added a comment - rebased the patch on the latest trunk, fixed the compilation issue
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 12s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
        0 mvndep 0m 32s Maven dependency ordering for branch
        +1 mvninstall 8m 24s trunk passed
        +1 compile 2m 48s trunk passed with JDK v1.8.0_91
        +1 compile 2m 41s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 41s trunk passed
        +1 mvnsite 1m 12s trunk passed
        +1 mvneclipse 0m 30s trunk passed
        -1 findbugs 1m 23s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common in trunk has 1 extant Findbugs warnings.
        +1 javadoc 1m 4s trunk passed with JDK v1.8.0_91
        +1 javadoc 1m 9s trunk passed with JDK v1.7.0_95
        0 mvndep 0m 13s Maven dependency ordering for patch
        +1 mvninstall 1m 3s the patch passed
        +1 compile 2m 45s the patch passed with JDK v1.8.0_91
        +1 cc 2m 45s the patch passed
        +1 javac 2m 45s the patch passed
        +1 compile 2m 40s the patch passed with JDK v1.7.0_95
        +1 cc 2m 40s the patch passed
        +1 javac 2m 40s the patch passed
        -1 checkstyle 0m 39s hadoop-yarn-project/hadoop-yarn: patch generated 2 new + 122 unchanged - 4 fixed = 124 total (was 126)
        +1 mvnsite 1m 9s the patch passed
        +1 mvneclipse 0m 26s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 53s the patch passed
        +1 javadoc 1m 1s the patch passed with JDK v1.8.0_91
        +1 javadoc 1m 5s the patch passed with JDK v1.7.0_95
        +1 unit 2m 40s hadoop-yarn-common in the patch passed with JDK v1.8.0_91.
        +1 unit 12m 31s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_91.
        +1 unit 2m 44s hadoop-yarn-common in the patch passed with JDK v1.7.0_95.
        +1 unit 12m 30s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_95.
        +1 asflicense 0m 24s Patch does not generate ASF License warnings.
        67m 52s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:cf2ee45
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12802516/yarn4766.004.patch
        JIRA Issue YARN-4766
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux 364f16b9c81d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / bb62e05
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        findbugs https://builds.apache.org/job/PreCommit-YARN-Build/11348/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-warnings.html
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11348/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11348/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/11348/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 32s Maven dependency ordering for branch +1 mvninstall 8m 24s trunk passed +1 compile 2m 48s trunk passed with JDK v1.8.0_91 +1 compile 2m 41s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 41s trunk passed +1 mvnsite 1m 12s trunk passed +1 mvneclipse 0m 30s trunk passed -1 findbugs 1m 23s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common in trunk has 1 extant Findbugs warnings. +1 javadoc 1m 4s trunk passed with JDK v1.8.0_91 +1 javadoc 1m 9s trunk passed with JDK v1.7.0_95 0 mvndep 0m 13s Maven dependency ordering for patch +1 mvninstall 1m 3s the patch passed +1 compile 2m 45s the patch passed with JDK v1.8.0_91 +1 cc 2m 45s the patch passed +1 javac 2m 45s the patch passed +1 compile 2m 40s the patch passed with JDK v1.7.0_95 +1 cc 2m 40s the patch passed +1 javac 2m 40s the patch passed -1 checkstyle 0m 39s hadoop-yarn-project/hadoop-yarn: patch generated 2 new + 122 unchanged - 4 fixed = 124 total (was 126) +1 mvnsite 1m 9s the patch passed +1 mvneclipse 0m 26s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 53s the patch passed +1 javadoc 1m 1s the patch passed with JDK v1.8.0_91 +1 javadoc 1m 5s the patch passed with JDK v1.7.0_95 +1 unit 2m 40s hadoop-yarn-common in the patch passed with JDK v1.8.0_91. +1 unit 12m 31s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_91. +1 unit 2m 44s hadoop-yarn-common in the patch passed with JDK v1.7.0_95. +1 unit 12m 30s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 24s Patch does not generate ASF License warnings. 67m 52s Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12802516/yarn4766.004.patch JIRA Issue YARN-4766 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 364f16b9c81d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / bb62e05 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-YARN-Build/11348/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-warnings.html checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11348/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11348/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/11348/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        haibochen Haibo Chen added a comment -

        findbug issue is unrelated to my patch

        Show
        haibochen Haibo Chen added a comment - findbug issue is unrelated to my patch
        Hide
        rchiang Ray Chiang added a comment -

        Minor nit: Rename noMoreRetentionLogFiles to obsoleteRetentionLogFIles. Similar rename for getNoMoreRetentionLogFiles().

        Show
        rchiang Ray Chiang added a comment - Minor nit: Rename noMoreRetentionLogFiles to obsoleteRetentionLogFIles. Similar rename for getNoMoreRetentionLogFiles().
        Hide
        haibochen Haibo Chen added a comment -

        Thanks Ray for your comment, the name you suggested is much better. Made the change and upload a new patch to verify the patch still works since it is been sitting for a while.

        Show
        haibochen Haibo Chen added a comment - Thanks Ray for your comment, the name you suggested is much better. Made the change and upload a new patch to verify the patch still works since it is been sitting for a while.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 10s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
        0 mvndep 0m 7s Maven dependency ordering for branch
        +1 mvninstall 6m 3s trunk passed
        +1 compile 1m 54s trunk passed
        +1 checkstyle 0m 40s trunk passed
        +1 mvnsite 0m 52s trunk passed
        +1 mvneclipse 0m 22s trunk passed
        +1 findbugs 1m 34s trunk passed
        +1 javadoc 0m 45s trunk passed
        0 mvndep 0m 8s Maven dependency ordering for patch
        +1 mvninstall 0m 44s the patch passed
        +1 compile 1m 51s the patch passed
        +1 cc 1m 51s the patch passed
        +1 javac 1m 51s the patch passed
        -1 checkstyle 0m 37s hadoop-yarn-project/hadoop-yarn: patch generated 26 new + 321 unchanged - 4 fixed = 347 total (was 325)
        +1 mvnsite 0m 48s the patch passed
        +1 mvneclipse 0m 18s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 1m 43s the patch passed
        -1 javadoc 1m 40s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 5 new + 592 unchanged - 0 fixed = 597 total (was 592)
        +1 javadoc 0m 41s the patch passed
        +1 unit 2m 5s hadoop-yarn-common in the patch passed.
        +1 unit 11m 16s hadoop-yarn-server-nodemanager in the patch passed.
        +1 asflicense 0m 16s Patch does not generate ASF License warnings.
        33m 40s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:2c91fd8
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12805063/yarn4766.005.patch
        JIRA Issue YARN-4766
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux 0e205f9dd54f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 22fcd81
        Default Java 1.8.0_91
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11571/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
        javadoc hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager: https://builds.apache.org/job/PreCommit-YARN-Build/11571/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11571/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/11571/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 7s Maven dependency ordering for branch +1 mvninstall 6m 3s trunk passed +1 compile 1m 54s trunk passed +1 checkstyle 0m 40s trunk passed +1 mvnsite 0m 52s trunk passed +1 mvneclipse 0m 22s trunk passed +1 findbugs 1m 34s trunk passed +1 javadoc 0m 45s trunk passed 0 mvndep 0m 8s Maven dependency ordering for patch +1 mvninstall 0m 44s the patch passed +1 compile 1m 51s the patch passed +1 cc 1m 51s the patch passed +1 javac 1m 51s the patch passed -1 checkstyle 0m 37s hadoop-yarn-project/hadoop-yarn: patch generated 26 new + 321 unchanged - 4 fixed = 347 total (was 325) +1 mvnsite 0m 48s the patch passed +1 mvneclipse 0m 18s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 43s the patch passed -1 javadoc 1m 40s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 5 new + 592 unchanged - 0 fixed = 597 total (was 592) +1 javadoc 0m 41s the patch passed +1 unit 2m 5s hadoop-yarn-common in the patch passed. +1 unit 11m 16s hadoop-yarn-server-nodemanager in the patch passed. +1 asflicense 0m 16s Patch does not generate ASF License warnings. 33m 40s Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12805063/yarn4766.005.patch JIRA Issue YARN-4766 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 0e205f9dd54f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 22fcd81 Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11571/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt javadoc hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager: https://builds.apache.org/job/PreCommit-YARN-Build/11571/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11571/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/11571/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        rkanter Robert Kanter added a comment -

        Haibo Chen, the 005 patch looks good to me. Can you take a look at the checkstyle and javadoc warnings?

        Show
        rkanter Robert Kanter added a comment - Haibo Chen , the 005 patch looks good to me. Can you take a look at the checkstyle and javadoc warnings?
        Hide
        haibochen Haibo Chen added a comment -

        The checkstyle warnings are against the protobuf-generated file. I'll try to clean up some of the javadoc warnings.

        Show
        haibochen Haibo Chen added a comment - The checkstyle warnings are against the protobuf-generated file. I'll try to clean up some of the javadoc warnings.
        Hide
        haibochen Haibo Chen added a comment -

        Rebase the patch on trunk, addressed some of the checkstyle issues. Many of them are related to accessibility of the fields. The fields are all final and the class to which the fields belong is private, so IMHO, the warnings are not necessarily valid. The javadoc issue is on a protobuf-generated file.

        Show
        haibochen Haibo Chen added a comment - Rebase the patch on trunk, addressed some of the checkstyle issues. Many of them are related to accessibility of the fields. The fields are all final and the class to which the fields belong is private, so IMHO, the warnings are not necessarily valid. The javadoc issue is on a protobuf-generated file.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
        0 mvndep 0m 10s Maven dependency ordering for branch
        +1 mvninstall 6m 44s trunk passed
        +1 compile 2m 6s trunk passed
        +1 checkstyle 0m 40s trunk passed
        +1 mvnsite 0m 58s trunk passed
        +1 mvneclipse 0m 24s trunk passed
        +1 findbugs 1m 36s trunk passed
        +1 javadoc 0m 49s trunk passed
        0 mvndep 0m 7s Maven dependency ordering for patch
        +1 mvninstall 0m 48s the patch passed
        +1 compile 1m 54s the patch passed
        +1 cc 1m 54s the patch passed
        +1 javac 1m 54s the patch passed
        -1 checkstyle 0m 42s hadoop-yarn-project/hadoop-yarn: patch generated 9 new + 319 unchanged - 2 fixed = 328 total (was 321)
        +1 mvnsite 0m 55s the patch passed
        +1 mvneclipse 0m 18s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 1m 48s the patch passed
        -1 javadoc 1m 43s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 5 new + 592 unchanged - 0 fixed = 597 total (was 592)
        +1 javadoc 0m 42s the patch passed
        +1 unit 2m 7s hadoop-yarn-common in the patch passed.
        +1 unit 11m 39s hadoop-yarn-server-nodemanager in the patch passed.
        +1 asflicense 0m 18s Patch does not generate ASF License warnings.
        35m 49s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:2c91fd8
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12805708/yarn4766.006.patch
        JIRA Issue YARN-4766
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux 552cedb1bbee 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / ac95448
        Default Java 1.8.0_91
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11636/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
        javadoc hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager: https://builds.apache.org/job/PreCommit-YARN-Build/11636/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11636/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/11636/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 10s Maven dependency ordering for branch +1 mvninstall 6m 44s trunk passed +1 compile 2m 6s trunk passed +1 checkstyle 0m 40s trunk passed +1 mvnsite 0m 58s trunk passed +1 mvneclipse 0m 24s trunk passed +1 findbugs 1m 36s trunk passed +1 javadoc 0m 49s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 0m 48s the patch passed +1 compile 1m 54s the patch passed +1 cc 1m 54s the patch passed +1 javac 1m 54s the patch passed -1 checkstyle 0m 42s hadoop-yarn-project/hadoop-yarn: patch generated 9 new + 319 unchanged - 2 fixed = 328 total (was 321) +1 mvnsite 0m 55s the patch passed +1 mvneclipse 0m 18s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 48s the patch passed -1 javadoc 1m 43s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 5 new + 592 unchanged - 0 fixed = 597 total (was 592) +1 javadoc 0m 42s the patch passed +1 unit 2m 7s hadoop-yarn-common in the patch passed. +1 unit 11m 39s hadoop-yarn-server-nodemanager in the patch passed. +1 asflicense 0m 18s Patch does not generate ASF License warnings. 35m 49s Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12805708/yarn4766.006.patch JIRA Issue YARN-4766 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 552cedb1bbee 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ac95448 Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11636/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt javadoc hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager: https://builds.apache.org/job/PreCommit-YARN-Build/11636/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11636/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/11636/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        rkanter Robert Kanter added a comment -

        +1 LGTM. Will commit sometime tomorrow morning.

        Show
        rkanter Robert Kanter added a comment - +1 LGTM. Will commit sometime tomorrow morning.
        Hide
        rkanter Robert Kanter added a comment -

        Thanks Haibo. Committed to trunk and branch-2!

        Show
        rkanter Robert Kanter added a comment - Thanks Haibo. Committed to trunk and branch-2!
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #9859 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9859/)
        YARN-4766. NM should not aggregate logs older than the retention policy (rkanter: rev e07519b8dbb96d73c48e910a4de12563c5c2f8aa)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/application/TestApplication.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/application/ApplicationImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/TestLogAggregationService.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/TestAppLogAggregatorImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/AggregatedLogFormat.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/proto/yarn_server_nodemanager_recovery.proto
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerAppStartedEvent.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/LogAggregationService.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #9859 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9859/ ) YARN-4766 . NM should not aggregate logs older than the retention policy (rkanter: rev e07519b8dbb96d73c48e910a4de12563c5c2f8aa) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/application/TestApplication.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/application/ApplicationImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/TestLogAggregationService.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/TestAppLogAggregatorImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/AggregatedLogFormat.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/proto/yarn_server_nodemanager_recovery.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerAppStartedEvent.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/LogAggregationService.java

          People

          • Assignee:
            haibochen Haibo Chen
            Reporter:
            haibochen Haibo Chen
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development