Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None

      Description

      We need to have an "offline" caching storage for timeline server v1.5 after the changes in YARN-3942. The in memory timeline storage may run into OOM issues when used as a cache storage for entity file timeline storage. We can refactor the code and have a level db based caching storage for this use case.

      1. YARN-4219-trunk.001.patch
        62 kB
        Li Lu
      2. YARN-4219-trunk.002.patch
        63 kB
        Li Lu
      3. YARN-4219-trunk.003.patch
        63 kB
        Li Lu
      4. YARN-4219-trunk.004.patch
        64 kB
        Li Lu
      5. YARN-4219-trunk.005.patch
        64 kB
        Li Lu
      6. YARN-4219-trunk.006.patch
        64 kB
        Li Lu
      7. YARN-4219-trunk.addendum1.patch
        1 kB
        Li Lu
      8. YARN-4219-YARN-4265.001.patch
        62 kB
        Li Lu
      9. YARN-4219-YARN-4265.002.patch
        64 kB
        Li Lu
      10. YARN-4219-YARN-4265.003.patch
        64 kB
        Li Lu

        Issue Links

          Activity

          Hide
          gtCarrera9 Li Lu added a comment -

          Upload a patch to add the leveldb cache storage. To organize this storage, I refactored MemoryTimelineStore, moved the abstract logic to operate on a general map of entities into a new MapTimelineStore. Both MemoryTimelineStore and LevelDbCacheTimelineStore are MapTimelineStores, communicating through the abstract logic through a TimelineStoreMapAdapter.

          Show
          gtCarrera9 Li Lu added a comment - Upload a patch to add the leveldb cache storage. To organize this storage, I refactored MemoryTimelineStore, moved the abstract logic to operate on a general map of entities into a new MapTimelineStore. Both MemoryTimelineStore and LevelDbCacheTimelineStore are MapTimelineStores, communicating through the abstract logic through a TimelineStoreMapAdapter.
          Hide
          gtCarrera9 Li Lu added a comment -

          Let me quickly flip the patch to address some findbugs warnings.

          Show
          gtCarrera9 Li Lu added a comment - Let me quickly flip the patch to address some findbugs warnings.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 16m 39s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          -1 javac 8m 10s The applied patch generated 2 additional warning messages.
          +1 javadoc 10m 26s There were no new javadoc warning messages.
          -1 release audit 0m 19s The applied patch generated 1 release audit warnings.
          -1 checkstyle 0m 30s The applied patch generated 15 new checkstyle issues (total was 59, now 74).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 32s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 1m 2s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 3m 31s Tests passed in hadoop-yarn-server-applicationhistoryservice.
              42m 47s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12766656/YARN-4219-trunk.001.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / be7a0ad
          javac https://builds.apache.org/job/PreCommit-YARN-Build/9447/artifact/patchprocess/diffJavacWarnings.txt
          Release Audit https://builds.apache.org/job/PreCommit-YARN-Build/9447/artifact/patchprocess/patchReleaseAuditProblems.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9447/artifact/patchprocess/diffcheckstylehadoop-yarn-server-applicationhistoryservice.txt
          hadoop-yarn-server-applicationhistoryservice test log https://builds.apache.org/job/PreCommit-YARN-Build/9447/artifact/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9447/testReport/
          Java 1.7.0_55
          uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9447/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 39s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. -1 javac 8m 10s The applied patch generated 2 additional warning messages. +1 javadoc 10m 26s There were no new javadoc warning messages. -1 release audit 0m 19s The applied patch generated 1 release audit warnings. -1 checkstyle 0m 30s The applied patch generated 15 new checkstyle issues (total was 59, now 74). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 32s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 1m 2s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 3m 31s Tests passed in hadoop-yarn-server-applicationhistoryservice.     42m 47s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12766656/YARN-4219-trunk.001.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / be7a0ad javac https://builds.apache.org/job/PreCommit-YARN-Build/9447/artifact/patchprocess/diffJavacWarnings.txt Release Audit https://builds.apache.org/job/PreCommit-YARN-Build/9447/artifact/patchprocess/patchReleaseAuditProblems.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9447/artifact/patchprocess/diffcheckstylehadoop-yarn-server-applicationhistoryservice.txt hadoop-yarn-server-applicationhistoryservice test log https://builds.apache.org/job/PreCommit-YARN-Build/9447/artifact/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9447/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9447/console This message was automatically generated.
          Hide
          gtCarrera9 Li Lu added a comment -

          New patch addressing Jenkins -1s.

          Show
          gtCarrera9 Li Lu added a comment - New patch addressing Jenkins -1s.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 7s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 3m 12s trunk passed
          +1 compile 0m 14s trunk passed with JDK v1.8.0_60
          +1 compile 0m 15s trunk passed with JDK v1.7.0_79
          +1 checkstyle 0m 9s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 0m 35s trunk passed
          +1 javadoc 0m 15s trunk passed with JDK v1.8.0_60
          +1 javadoc 0m 19s trunk passed with JDK v1.7.0_79
          +1 mvninstall 0m 18s the patch passed
          +1 compile 0m 14s the patch passed with JDK v1.8.0_60
          -1 javac 1m 42s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice-jdk1.8.0_60 with JDK v1.8.0_60 has problems.
          +1 javac 0m 14s the patch passed
          +1 compile 0m 15s the patch passed with JDK v1.7.0_79
          -1 javac 1m 58s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice-jdk1.7.0_79 with JDK v1.7.0_79 has problems.
          +1 javac 0m 15s the patch passed
          -1 checkstyle 0m 9s Patch generated 5 new checkstyle issues in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice (total was 59, now 64).
          +1 mvneclipse 0m 15s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 findbugs 0m 46s the patch passed
          -1 javadoc 0m 16s hadoop-yarn-server-applicationhistoryservice in the patch failed with JDK v1.8.0_60.
          +1 javadoc 0m 19s the patch passed with JDK v1.7.0_79
          +1 unit 3m 9s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.8.0_60.
          +1 unit 3m 24s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.7.0_79.
          +1 asflicense 0m 23s Patch does not generate ASF License warnings.
          15m 47s



          Subsystem Report/Notes
          Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-10-30
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12769674/YARN-4219-trunk.002.patch
          JIRA Issue YARN-4219
          Optional Tests asflicense javac javadoc mvninstall unit xml findbugs checkstyle compile
          uname Linux 943ee53b4e79 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/apache-yetus-e77b1ce/precommit/personality/hadoop.sh
          git revision trunk / 7412ff4
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          findbugs v3.0.0
          javac hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice-jdk1.8.0_60: https://builds.apache.org/job/PreCommit-YARN-Build/9605/artifact/patchprocess/javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice-jdk1.8.0_60-diff.txt
          javac hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice-jdk1.7.0_79: https://builds.apache.org/job/PreCommit-YARN-Build/9605/artifact/patchprocess/javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice-jdk1.7.0_79-diff.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9605/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/9605/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice-jdk1.8.0_60.txt
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9605/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice
          Max memory used 228MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9605/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 7s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 3m 12s trunk passed +1 compile 0m 14s trunk passed with JDK v1.8.0_60 +1 compile 0m 15s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 9s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 0m 35s trunk passed +1 javadoc 0m 15s trunk passed with JDK v1.8.0_60 +1 javadoc 0m 19s trunk passed with JDK v1.7.0_79 +1 mvninstall 0m 18s the patch passed +1 compile 0m 14s the patch passed with JDK v1.8.0_60 -1 javac 1m 42s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice-jdk1.8.0_60 with JDK v1.8.0_60 has problems. +1 javac 0m 14s the patch passed +1 compile 0m 15s the patch passed with JDK v1.7.0_79 -1 javac 1m 58s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice-jdk1.7.0_79 with JDK v1.7.0_79 has problems. +1 javac 0m 15s the patch passed -1 checkstyle 0m 9s Patch generated 5 new checkstyle issues in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice (total was 59, now 64). +1 mvneclipse 0m 15s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 findbugs 0m 46s the patch passed -1 javadoc 0m 16s hadoop-yarn-server-applicationhistoryservice in the patch failed with JDK v1.8.0_60. +1 javadoc 0m 19s the patch passed with JDK v1.7.0_79 +1 unit 3m 9s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.8.0_60. +1 unit 3m 24s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.7.0_79. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 15m 47s Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-10-30 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12769674/YARN-4219-trunk.002.patch JIRA Issue YARN-4219 Optional Tests asflicense javac javadoc mvninstall unit xml findbugs checkstyle compile uname Linux 943ee53b4e79 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/apache-yetus-e77b1ce/precommit/personality/hadoop.sh git revision trunk / 7412ff4 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 javac hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice-jdk1.8.0_60: https://builds.apache.org/job/PreCommit-YARN-Build/9605/artifact/patchprocess/javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice-jdk1.8.0_60-diff.txt javac hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice-jdk1.7.0_79: https://builds.apache.org/job/PreCommit-YARN-Build/9605/artifact/patchprocess/javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice-jdk1.7.0_79-diff.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9605/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/9605/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice-jdk1.8.0_60.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9605/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice Max memory used 228MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/9605/console This message was automatically generated.
          Hide
          gtCarrera9 Li Lu added a comment -

          Attaching 003 patch to fix javac warnings and javadoc errors.

          Show
          gtCarrera9 Li Lu added a comment - Attaching 003 patch to fix javac warnings and javadoc errors.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 10s docker + precommit patch detected.
          +1 @author 0m 1s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 3m 34s trunk passed
          +1 compile 0m 18s trunk passed with JDK v1.8.0_60
          +1 compile 0m 16s trunk passed with JDK v1.7.0_79
          +1 checkstyle 0m 11s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 0m 42s trunk passed
          +1 javadoc 0m 19s trunk passed with JDK v1.8.0_60
          +1 javadoc 0m 19s trunk passed with JDK v1.7.0_79
          +1 mvninstall 0m 20s the patch passed
          +1 compile 0m 15s the patch passed with JDK v1.8.0_60
          +1 javac 0m 15s the patch passed
          +1 compile 0m 16s the patch passed with JDK v1.7.0_79
          +1 javac 0m 16s the patch passed
          -1 checkstyle 0m 11s Patch generated 4 new checkstyle issues in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice (total was 59, now 63).
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 0m 50s the patch passed
          +1 javadoc 0m 17s the patch passed with JDK v1.8.0_60
          +1 javadoc 0m 19s the patch passed with JDK v1.7.0_79
          +1 unit 3m 20s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.8.0_60.
          +1 unit 3m 43s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.7.0_79.
          +1 asflicense 0m 27s Patch does not generate ASF License warnings.
          17m 21s



          Subsystem Report/Notes
          Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-05
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12770890/YARN-4219-trunk.003.patch
          JIRA Issue YARN-4219
          Optional Tests asflicense javac javadoc mvninstall unit xml compile findbugs checkstyle
          uname Linux 2d0c4cb2ffed 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh
          git revision trunk / 286cc64
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9633/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9633/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice
          Max memory used 227MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9633/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s docker + precommit patch detected. +1 @author 0m 1s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 3m 34s trunk passed +1 compile 0m 18s trunk passed with JDK v1.8.0_60 +1 compile 0m 16s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 11s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 0m 42s trunk passed +1 javadoc 0m 19s trunk passed with JDK v1.8.0_60 +1 javadoc 0m 19s trunk passed with JDK v1.7.0_79 +1 mvninstall 0m 20s the patch passed +1 compile 0m 15s the patch passed with JDK v1.8.0_60 +1 javac 0m 15s the patch passed +1 compile 0m 16s the patch passed with JDK v1.7.0_79 +1 javac 0m 16s the patch passed -1 checkstyle 0m 11s Patch generated 4 new checkstyle issues in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice (total was 59, now 63). +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 0m 50s the patch passed +1 javadoc 0m 17s the patch passed with JDK v1.8.0_60 +1 javadoc 0m 19s the patch passed with JDK v1.7.0_79 +1 unit 3m 20s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.8.0_60. +1 unit 3m 43s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.7.0_79. +1 asflicense 0m 27s Patch does not generate ASF License warnings. 17m 21s Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-05 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12770890/YARN-4219-trunk.003.patch JIRA Issue YARN-4219 Optional Tests asflicense javac javadoc mvninstall unit xml compile findbugs checkstyle uname Linux 2d0c4cb2ffed 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh git revision trunk / 286cc64 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9633/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9633/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice Max memory used 227MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/9633/console This message was automatically generated.
          Hide
          xgong Xuan Gong added a comment -

          +1. The last patch looks good to me. Let us wait for several days. If there are no other comments, I will commit this later.

          Jason Lowe and Jonathan Eagles Do we have any comments for this ?

          Show
          xgong Xuan Gong added a comment - +1. The last patch looks good to me. Let us wait for several days. If there are no other comments, I will commit this later. Jason Lowe and Jonathan Eagles Do we have any comments for this ?
          Hide
          jlowe Jason Lowe added a comment -

          Shouldn't we commit some form of YARN-3942 before committing this? I got the impression this was an underlying storage to be used by the EntityFileTimelineStore, or is there a use-case for this outside of the v1.5 core code in YARN-3942?

          Patch looks pretty good except for one thing I noticed: The pom file should not have hardcoded versions in it. It should omit the versions and leave it up to hadoop-project/pom.xml to define that. Otherwise we risk having different portions of code needing different versions of the same dependency.

          Show
          jlowe Jason Lowe added a comment - Shouldn't we commit some form of YARN-3942 before committing this? I got the impression this was an underlying storage to be used by the EntityFileTimelineStore, or is there a use-case for this outside of the v1.5 core code in YARN-3942 ? Patch looks pretty good except for one thing I noticed: The pom file should not have hardcoded versions in it. It should omit the versions and leave it up to hadoop-project/pom.xml to define that. Otherwise we risk having different portions of code needing different versions of the same dependency.
          Hide
          gtCarrera9 Li Lu added a comment -

          Yes, this is a caching storage for ATS v1.5 design. We can prioritize any "up-level" storage that uses this caching storage. I'll fix the maven problem soon.

          Show
          gtCarrera9 Li Lu added a comment - Yes, this is a caching storage for ATS v1.5 design. We can prioritize any "up-level" storage that uses this caching storage. I'll fix the maven problem soon.
          Hide
          gtCarrera9 Li Lu added a comment -

          I rebased my YARN-4219 patch on top of YARN-4265 since the caching storage is an extension on the main plugin storage. I also changed some class names to better reflect their implementations. The current patch applies on top of YARN-4265, which on top of YARN-4234.

          Show
          gtCarrera9 Li Lu added a comment - I rebased my YARN-4219 patch on top of YARN-4265 since the caching storage is an extension on the main plugin storage. I also changed some class names to better reflect their implementations. The current patch applies on top of YARN-4265 , which on top of YARN-4234 .
          Hide
          gtCarrera9 Li Lu added a comment -

          I noticed one problem for the current ATS v1.5's caching policy is that a cache storage may still be accessed after the cache is evicted. A quick fix is on the storage side: we can synchronously stop the storage service, and check if the service is already stopped on each storage request. We can build a more graceful cache evicting mechanism in future.

          Show
          gtCarrera9 Li Lu added a comment - I noticed one problem for the current ATS v1.5's caching policy is that a cache storage may still be accessed after the cache is evicted. A quick fix is on the storage side: we can synchronously stop the storage service, and check if the service is already stopped on each storage request. We can build a more graceful cache evicting mechanism in future.
          Hide
          gtCarrera9 Li Lu added a comment -

          Fixed the synchronization issue on service stop.

          Show
          gtCarrera9 Li Lu added a comment - Fixed the synchronization issue on service stop.
          Hide
          gtCarrera9 Li Lu added a comment -

          Rebase the patch to latest trunk.

          Show
          gtCarrera9 Li Lu added a comment - Rebase the patch to latest trunk.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          0 mvndep 0m 33s Maven dependency ordering for branch
          +1 mvninstall 7m 46s trunk passed
          +1 compile 2m 15s trunk passed with JDK v1.8.0_66
          +1 compile 2m 10s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 28s trunk passed
          +1 mvnsite 1m 47s trunk passed
          +1 mvneclipse 0m 50s trunk passed
          +1 findbugs 3m 47s trunk passed
          +1 javadoc 1m 39s trunk passed with JDK v1.8.0_66
          +1 javadoc 4m 4s trunk passed with JDK v1.7.0_91
          0 mvndep 0m 38s Maven dependency ordering for patch
          +1 mvninstall 1m 32s the patch passed
          +1 compile 1m 45s the patch passed with JDK v1.8.0_66
          +1 javac 1m 45s the patch passed
          +1 compile 2m 9s the patch passed with JDK v1.7.0_91
          +1 javac 2m 9s the patch passed
          -1 checkstyle 0m 30s hadoop-yarn-project/hadoop-yarn: patch generated 12 new + 225 unchanged - 1 fixed = 237 total (was 226)
          +1 mvnsite 1m 39s the patch passed
          +1 mvneclipse 0m 46s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          -1 findbugs 0m 50s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
          -1 findbugs 0m 50s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
          +1 javadoc 1m 25s the patch passed with JDK v1.8.0_66
          +1 javadoc 3m 50s the patch passed with JDK v1.7.0_91
          +1 unit 0m 20s hadoop-yarn-api in the patch passed with JDK v1.8.0_66.
          +1 unit 1m 50s hadoop-yarn-common in the patch passed with JDK v1.8.0_66.
          +1 unit 3m 31s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.8.0_66.
          +1 unit 3m 34s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.8.0_66.
          +1 unit 0m 23s hadoop-yarn-api in the patch passed with JDK v1.7.0_91.
          +1 unit 2m 12s hadoop-yarn-common in the patch passed with JDK v1.7.0_91.
          +1 unit 4m 2s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.7.0_91.
          +1 unit 3m 52s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.7.0_91.
          +1 asflicense 0m 19s Patch does not generate ASF License warnings.
          65m 56s



          Reason Tests
          FindBugs module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice
            Inconsistent synchronization of org.apache.hadoop.yarn.server.timeline.KeyValueBasedTimelineStore.serviceStopped; locked 62% of time Unsynchronized access at KeyValueBasedTimelineStore.java:62% of time Unsynchronized access at KeyValueBasedTimelineStore.java:[line 276]
          FindBugs module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice
            Inconsistent synchronization of org.apache.hadoop.yarn.server.timeline.KeyValueBasedTimelineStore.serviceStopped; locked 62% of time Unsynchronized access at KeyValueBasedTimelineStore.java:62% of time Unsynchronized access at KeyValueBasedTimelineStore.java:[line 276]



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782926/YARN-4219-trunk.004.patch
          JIRA Issue YARN-4219
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux ab76f5b50e18 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 8bc93db
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10321/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
          findbugs https://builds.apache.org/job/PreCommit-YARN-Build/10321/artifact/patchprocess/new-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.html
          findbugs https://builds.apache.org/job/PreCommit-YARN-Build/10321/artifact/patchprocess/new-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.html
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10321/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage U: hadoop-yarn-project/hadoop-yarn
          Max memory used 77MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10321/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 33s Maven dependency ordering for branch +1 mvninstall 7m 46s trunk passed +1 compile 2m 15s trunk passed with JDK v1.8.0_66 +1 compile 2m 10s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 28s trunk passed +1 mvnsite 1m 47s trunk passed +1 mvneclipse 0m 50s trunk passed +1 findbugs 3m 47s trunk passed +1 javadoc 1m 39s trunk passed with JDK v1.8.0_66 +1 javadoc 4m 4s trunk passed with JDK v1.7.0_91 0 mvndep 0m 38s Maven dependency ordering for patch +1 mvninstall 1m 32s the patch passed +1 compile 1m 45s the patch passed with JDK v1.8.0_66 +1 javac 1m 45s the patch passed +1 compile 2m 9s the patch passed with JDK v1.7.0_91 +1 javac 2m 9s the patch passed -1 checkstyle 0m 30s hadoop-yarn-project/hadoop-yarn: patch generated 12 new + 225 unchanged - 1 fixed = 237 total (was 226) +1 mvnsite 1m 39s the patch passed +1 mvneclipse 0m 46s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. -1 findbugs 0m 50s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) -1 findbugs 0m 50s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) +1 javadoc 1m 25s the patch passed with JDK v1.8.0_66 +1 javadoc 3m 50s the patch passed with JDK v1.7.0_91 +1 unit 0m 20s hadoop-yarn-api in the patch passed with JDK v1.8.0_66. +1 unit 1m 50s hadoop-yarn-common in the patch passed with JDK v1.8.0_66. +1 unit 3m 31s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.8.0_66. +1 unit 3m 34s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.8.0_66. +1 unit 0m 23s hadoop-yarn-api in the patch passed with JDK v1.7.0_91. +1 unit 2m 12s hadoop-yarn-common in the patch passed with JDK v1.7.0_91. +1 unit 4m 2s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.7.0_91. +1 unit 3m 52s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 19s Patch does not generate ASF License warnings. 65m 56s Reason Tests FindBugs module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice   Inconsistent synchronization of org.apache.hadoop.yarn.server.timeline.KeyValueBasedTimelineStore.serviceStopped; locked 62% of time Unsynchronized access at KeyValueBasedTimelineStore.java:62% of time Unsynchronized access at KeyValueBasedTimelineStore.java: [line 276] FindBugs module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice   Inconsistent synchronization of org.apache.hadoop.yarn.server.timeline.KeyValueBasedTimelineStore.serviceStopped; locked 62% of time Unsynchronized access at KeyValueBasedTimelineStore.java:62% of time Unsynchronized access at KeyValueBasedTimelineStore.java: [line 276] Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782926/YARN-4219-trunk.004.patch JIRA Issue YARN-4219 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux ab76f5b50e18 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 8bc93db Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10321/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt findbugs https://builds.apache.org/job/PreCommit-YARN-Build/10321/artifact/patchprocess/new-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.html findbugs https://builds.apache.org/job/PreCommit-YARN-Build/10321/artifact/patchprocess/new-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.html JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10321/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage U: hadoop-yarn-project/hadoop-yarn Max memory used 77MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/10321/console This message was automatically generated.
          Hide
          xgong Xuan Gong added a comment -

          Thanks for the patch. Li Lu.

          The latest patch looks good to me. Can we fix the findbugs warning, please ?

          Jason Lowe Do you have any other comments ?

          Show
          xgong Xuan Gong added a comment - Thanks for the patch. Li Lu . The latest patch looks good to me. Can we fix the findbugs warning, please ? Jason Lowe Do you have any other comments ?
          Hide
          gtCarrera9 Li Lu added a comment -

          Thanks Xuan Gong for the comment! Upload a new patch to address the findbugs warning.

          Show
          gtCarrera9 Li Lu added a comment - Thanks Xuan Gong for the comment! Upload a new patch to address the findbugs warning.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          0 mvndep 1m 3s Maven dependency ordering for branch
          +1 mvninstall 11m 4s trunk passed
          +1 compile 3m 35s trunk passed with JDK v1.8.0_66
          +1 compile 3m 6s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 43s trunk passed
          +1 mvnsite 2m 28s trunk passed
          +1 mvneclipse 1m 14s trunk passed
          +1 findbugs 5m 21s trunk passed
          +1 javadoc 2m 41s trunk passed with JDK v1.8.0_66
          +1 javadoc 5m 13s trunk passed with JDK v1.7.0_91
          0 mvndep 0m 45s Maven dependency ordering for patch
          +1 mvninstall 2m 1s the patch passed
          +1 compile 3m 25s the patch passed with JDK v1.8.0_66
          +1 javac 3m 25s the patch passed
          +1 compile 3m 4s the patch passed with JDK v1.7.0_91
          +1 javac 3m 4s the patch passed
          -1 checkstyle 0m 42s hadoop-yarn-project/hadoop-yarn: patch generated 12 new + 225 unchanged - 1 fixed = 237 total (was 226)
          +1 mvnsite 2m 18s the patch passed
          +1 mvneclipse 1m 4s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 6m 17s the patch passed
          +1 javadoc 2m 29s the patch passed with JDK v1.8.0_66
          +1 javadoc 4m 56s the patch passed with JDK v1.7.0_91
          +1 unit 0m 36s hadoop-yarn-api in the patch passed with JDK v1.8.0_66.
          +1 unit 2m 54s hadoop-yarn-common in the patch passed with JDK v1.8.0_66.
          +1 unit 4m 8s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.8.0_66.
          +1 unit 4m 5s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.8.0_66.
          +1 unit 0m 33s hadoop-yarn-api in the patch passed with JDK v1.7.0_91.
          +1 unit 2m 45s hadoop-yarn-common in the patch passed with JDK v1.7.0_91.
          +1 unit 3m 54s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.7.0_91.
          +1 unit 3m 55s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.7.0_91.
          +1 asflicense 0m 27s Patch does not generate ASF License warnings.
          89m 34s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12783007/YARN-4219-trunk.005.patch
          JIRA Issue YARN-4219
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 6dc60baba0c3 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / edc43a9
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10325/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10325/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage U: hadoop-yarn-project/hadoop-yarn
          Max memory used 77MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10325/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 1m 3s Maven dependency ordering for branch +1 mvninstall 11m 4s trunk passed +1 compile 3m 35s trunk passed with JDK v1.8.0_66 +1 compile 3m 6s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 43s trunk passed +1 mvnsite 2m 28s trunk passed +1 mvneclipse 1m 14s trunk passed +1 findbugs 5m 21s trunk passed +1 javadoc 2m 41s trunk passed with JDK v1.8.0_66 +1 javadoc 5m 13s trunk passed with JDK v1.7.0_91 0 mvndep 0m 45s Maven dependency ordering for patch +1 mvninstall 2m 1s the patch passed +1 compile 3m 25s the patch passed with JDK v1.8.0_66 +1 javac 3m 25s the patch passed +1 compile 3m 4s the patch passed with JDK v1.7.0_91 +1 javac 3m 4s the patch passed -1 checkstyle 0m 42s hadoop-yarn-project/hadoop-yarn: patch generated 12 new + 225 unchanged - 1 fixed = 237 total (was 226) +1 mvnsite 2m 18s the patch passed +1 mvneclipse 1m 4s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 6m 17s the patch passed +1 javadoc 2m 29s the patch passed with JDK v1.8.0_66 +1 javadoc 4m 56s the patch passed with JDK v1.7.0_91 +1 unit 0m 36s hadoop-yarn-api in the patch passed with JDK v1.8.0_66. +1 unit 2m 54s hadoop-yarn-common in the patch passed with JDK v1.8.0_66. +1 unit 4m 8s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.8.0_66. +1 unit 4m 5s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.8.0_66. +1 unit 0m 33s hadoop-yarn-api in the patch passed with JDK v1.7.0_91. +1 unit 2m 45s hadoop-yarn-common in the patch passed with JDK v1.7.0_91. +1 unit 3m 54s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.7.0_91. +1 unit 3m 55s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 27s Patch does not generate ASF License warnings. 89m 34s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12783007/YARN-4219-trunk.005.patch JIRA Issue YARN-4219 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 6dc60baba0c3 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / edc43a9 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10325/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10325/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage U: hadoop-yarn-project/hadoop-yarn Max memory used 77MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/10325/console This message was automatically generated.
          Hide
          djp Junping Du added a comment -

          Thanks Li Lu for updating the patch and Xuan Gong for review.
          I briefly go through the patch, it looks fine in overall but could be some trivial issues.
          A quick question for MapTimelineStore, what's this method doing which is a bit confusing?

          +  private static Object maybeConvert(Object o) {
          +    if (o instanceof Long) {
          +      Long l = (Long)o;
          +      if (l >= Integer.MIN_VALUE && l <= Integer.MAX_VALUE) {
          +        return l.intValue();
          +      }
          +    }
          +    return o;
          +  }
          
          Show
          djp Junping Du added a comment - Thanks Li Lu for updating the patch and Xuan Gong for review. I briefly go through the patch, it looks fine in overall but could be some trivial issues. A quick question for MapTimelineStore, what's this method doing which is a bit confusing? + private static Object maybeConvert(Object o) { + if (o instanceof Long) { + Long l = (Long)o; + if (l >= Integer.MIN_VALUE && l <= Integer.MAX_VALUE) { + return l.intValue(); + } + } + return o; + }
          Hide
          gtCarrera9 Li Lu added a comment -

          Thanks Junping for the review! The method you pointed out is to compact a number object, and try to find the best way to store it (Integer or Long). The name is a little bit confusing so I renamed it. Also, I slightly refactored the util methods to put them in a nested util class.

          Show
          gtCarrera9 Li Lu added a comment - Thanks Junping for the review! The method you pointed out is to compact a number object, and try to find the best way to store it (Integer or Long). The name is a little bit confusing so I renamed it. Also, I slightly refactored the util methods to put them in a nested util class.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          0 mvndep 0m 39s Maven dependency ordering for branch
          +1 mvninstall 8m 19s trunk passed
          +1 compile 2m 3s trunk passed with JDK v1.8.0_66
          +1 compile 2m 15s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 33s trunk passed
          +1 mvnsite 1m 54s trunk passed
          +1 mvneclipse 0m 55s trunk passed
          +1 findbugs 4m 5s trunk passed
          +1 javadoc 1m 43s trunk passed with JDK v1.8.0_66
          +1 javadoc 4m 12s trunk passed with JDK v1.7.0_91
          0 mvndep 0m 34s Maven dependency ordering for patch
          +1 mvninstall 1m 32s the patch passed
          +1 compile 1m 56s the patch passed with JDK v1.8.0_66
          +1 javac 1m 56s the patch passed
          +1 compile 2m 15s the patch passed with JDK v1.7.0_91
          +1 javac 2m 15s the patch passed
          -1 checkstyle 0m 32s hadoop-yarn-project/hadoop-yarn: patch generated 12 new + 225 unchanged - 1 fixed = 237 total (was 226)
          +1 mvnsite 1m 46s the patch passed
          +1 mvneclipse 0m 47s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 4m 52s the patch passed
          +1 javadoc 1m 34s the patch passed with JDK v1.8.0_66
          +1 javadoc 4m 4s the patch passed with JDK v1.7.0_91
          +1 unit 0m 24s hadoop-yarn-api in the patch passed with JDK v1.8.0_66.
          +1 unit 1m 59s hadoop-yarn-common in the patch passed with JDK v1.8.0_66.
          +1 unit 3m 4s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.8.0_66.
          +1 unit 3m 2s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.8.0_66.
          +1 unit 0m 23s hadoop-yarn-api in the patch passed with JDK v1.7.0_91.
          +1 unit 2m 15s hadoop-yarn-common in the patch passed with JDK v1.7.0_91.
          +1 unit 3m 17s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.7.0_91.
          +1 unit 3m 16s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.7.0_91.
          +1 asflicense 0m 21s Patch does not generate ASF License warnings.
          66m 18s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12783951/YARN-4219-trunk.006.patch
          JIRA Issue YARN-4219
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux c76c597ed7e6 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 46e5ea8
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10373/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10373/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage U: hadoop-yarn-project/hadoop-yarn
          Max memory used 76MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10373/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 39s Maven dependency ordering for branch +1 mvninstall 8m 19s trunk passed +1 compile 2m 3s trunk passed with JDK v1.8.0_66 +1 compile 2m 15s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 33s trunk passed +1 mvnsite 1m 54s trunk passed +1 mvneclipse 0m 55s trunk passed +1 findbugs 4m 5s trunk passed +1 javadoc 1m 43s trunk passed with JDK v1.8.0_66 +1 javadoc 4m 12s trunk passed with JDK v1.7.0_91 0 mvndep 0m 34s Maven dependency ordering for patch +1 mvninstall 1m 32s the patch passed +1 compile 1m 56s the patch passed with JDK v1.8.0_66 +1 javac 1m 56s the patch passed +1 compile 2m 15s the patch passed with JDK v1.7.0_91 +1 javac 2m 15s the patch passed -1 checkstyle 0m 32s hadoop-yarn-project/hadoop-yarn: patch generated 12 new + 225 unchanged - 1 fixed = 237 total (was 226) +1 mvnsite 1m 46s the patch passed +1 mvneclipse 0m 47s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 4m 52s the patch passed +1 javadoc 1m 34s the patch passed with JDK v1.8.0_66 +1 javadoc 4m 4s the patch passed with JDK v1.7.0_91 +1 unit 0m 24s hadoop-yarn-api in the patch passed with JDK v1.8.0_66. +1 unit 1m 59s hadoop-yarn-common in the patch passed with JDK v1.8.0_66. +1 unit 3m 4s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.8.0_66. +1 unit 3m 2s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.8.0_66. +1 unit 0m 23s hadoop-yarn-api in the patch passed with JDK v1.7.0_91. +1 unit 2m 15s hadoop-yarn-common in the patch passed with JDK v1.7.0_91. +1 unit 3m 17s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.7.0_91. +1 unit 3m 16s hadoop-yarn-server-applicationhistoryservice in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 21s Patch does not generate ASF License warnings. 66m 18s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12783951/YARN-4219-trunk.006.patch JIRA Issue YARN-4219 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux c76c597ed7e6 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 46e5ea8 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10373/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10373/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage U: hadoop-yarn-project/hadoop-yarn Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/10373/console This message was automatically generated.
          Hide
          jlowe Jason Lowe added a comment -

          +1, latest patch lgtm.

          Show
          jlowe Jason Lowe added a comment - +1, latest patch lgtm.
          Hide
          djp Junping Du added a comment -

          +1. Latest patch LGTM too.

          Show
          djp Junping Du added a comment - +1. Latest patch LGTM too.
          Hide
          xgong Xuan Gong added a comment -

          Committed into trunk/branch-2/branch-2.8. Thanks, Li for the patch. And Thanks for the review, Junping and Jason

          Show
          xgong Xuan Gong added a comment - Committed into trunk/branch-2/branch-2.8. Thanks, Li for the patch. And Thanks for the review, Junping and Jason
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9202 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9202/)
          YARN-4219. New levelDB cache storage for timeline v1.5. Contributed by (xgong: rev 9fab22b36673e7f1a0bb629d2c07966ac2482e99)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/src/test/java/org/apache/hadoop/yarn/server/timeline/TestLevelDBCacheTimelineStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/util/LeveldbUtils.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/src/main/java/org/apache/hadoop/yarn/server/timeline/EntityCacheItem.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/pom.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/TimelineStoreMapAdapter.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/KeyValueBasedTimelineStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/src/test/java/org/apache/hadoop/yarn/server/timeline/PluginStoreTestUtils.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/MemoryTimelineStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/src/main/java/org/apache/hadoop/yarn/server/timeline/LevelDBCacheTimelineStore.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9202 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9202/ ) YARN-4219 . New levelDB cache storage for timeline v1.5. Contributed by (xgong: rev 9fab22b36673e7f1a0bb629d2c07966ac2482e99) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/src/test/java/org/apache/hadoop/yarn/server/timeline/TestLevelDBCacheTimelineStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/util/LeveldbUtils.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/src/main/java/org/apache/hadoop/yarn/server/timeline/EntityCacheItem.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/pom.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/TimelineStoreMapAdapter.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/KeyValueBasedTimelineStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/src/test/java/org/apache/hadoop/yarn/server/timeline/PluginStoreTestUtils.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/MemoryTimelineStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/src/main/java/org/apache/hadoop/yarn/server/timeline/LevelDBCacheTimelineStore.java
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          It seems this check in failing the javadoc compilation build

          Show
          rohithsharma Rohith Sharma K S added a comment - It seems this check in failing the javadoc compilation build
          Hide
          gtCarrera9 Li Lu added a comment -

          OK, let me quickly post a fix.

          Show
          gtCarrera9 Li Lu added a comment - OK, let me quickly post a fix.
          Hide
          gtCarrera9 Li Lu added a comment -

          Upload an addendum patch to fix the javadoc issue with Java 8.

          Show
          gtCarrera9 Li Lu added a comment - Upload an addendum patch to fix the javadoc issue with Java 8.
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          OK, I didn't see your reply, before that I raised YARN-4656!!

          Show
          rohithsharma Rohith Sharma K S added a comment - OK, I didn't see your reply, before that I raised YARN-4656 !!
          Hide
          gtCarrera9 Li Lu added a comment -

          NP, let's address this problem within 4219.

          Show
          gtCarrera9 Li Lu added a comment - NP, let's address this problem within 4219.
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          Thanks for providing quick fix. I am compiling javadoc in java-1.8. I will commit it after my local compilation is success!!

          Show
          rohithsharma Rohith Sharma K S added a comment - Thanks for providing quick fix. I am compiling javadoc in java-1.8. I will commit it after my local compilation is success!!
          Hide
          gtCarrera9 Li Lu added a comment -

          Thanks Rohith Sharma K S! Let me know if there's any problem.

          Show
          gtCarrera9 Li Lu added a comment - Thanks Rohith Sharma K S ! Let me know if there's any problem.
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          Compilation is success, committing in.

          Show
          rohithsharma Rohith Sharma K S added a comment - Compilation is success, committing in.
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          committed to trunk/branch-2/branch-2.8.. thanks Li Lu for quick fix

          Show
          rohithsharma Rohith Sharma K S added a comment - committed to trunk/branch-2/branch-2.8.. thanks Li Lu for quick fix
          Hide
          gtCarrera9 Li Lu added a comment -
          Show
          gtCarrera9 Li Lu added a comment - Thanks Rohith Sharma K S !
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9205 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9205/)
          YARN-4219. addendum patch to fix javadoc errors (rohithsharmaks: rev 09d831c95ba18e2892cddd749f6e06f112dda7f5)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/src/main/java/org/apache/hadoop/yarn/server/timeline/LevelDBCacheTimelineStore.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9205 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9205/ ) YARN-4219 . addendum patch to fix javadoc errors (rohithsharmaks: rev 09d831c95ba18e2892cddd749f6e06f112dda7f5) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/src/main/java/org/apache/hadoop/yarn/server/timeline/LevelDBCacheTimelineStore.java

            People

            • Assignee:
              gtCarrera9 Li Lu
              Reporter:
              gtCarrera9 Li Lu
            • Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development