Details

    • Hadoop Flags:
      Reviewed

      Description

      lWhen the JHS starts up, it initializes the internal caches and storage via the HistoryFileManager. If we have a large number of existing finished jobs then we could spent minutes in this startup phase without logging progress:
      2016-03-14 10:56:01,444 INFO org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://hadoopcdh.itnas01.ieee.org:8020]
      2016-03-14 10:56:11,455 INFO org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager: Initializing Existing Jobs...
      2016-03-14 12:01:36,926 INFO org.apache.hadoop.mapreduce.v2.hs.CachedHistoryStorage: CachedHistoryStorage Init
      This makes it really difficult to assess if things are working correctly (it looks hung). We can add logs to notify users of progress.

      1. mapreduce6718.001.patch
        2 kB
        Haibo Chen
      2. mapreduce6718.002.patch
        2 kB
        Haibo Chen
      3. mapreduce6718.003.patch
        3 kB
        Haibo Chen

        Activity

        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10509 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10509/)
        MAPREDUCE-6718. add progress log to JHS during startup (haibochen via (rkanter: rev 0d6778d800ff16366911e3b064f3af6162dee2e4)

        • (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryFileManager.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10509 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10509/ ) MAPREDUCE-6718 . add progress log to JHS during startup (haibochen via (rkanter: rev 0d6778d800ff16366911e3b064f3af6162dee2e4) (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryFileManager.java
        Hide
        rkanter Robert Kanter added a comment -

        Thanks Haibo Chen. Committed to trunk and branch-2!

        Show
        rkanter Robert Kanter added a comment - Thanks Haibo Chen . Committed to trunk and branch-2!
        Hide
        rkanter Robert Kanter added a comment -

        +1

        Show
        rkanter Robert Kanter added a comment - +1
        Hide
        rkanter Robert Kanter added a comment -

        You're right. That should have gone on MAPREDUCE-6765

        Show
        rkanter Robert Kanter added a comment - You're right. That should have gone on MAPREDUCE-6765
        Hide
        haibochen Haibo Chen added a comment -

        Commented on the wrong jira?

        Show
        haibochen Haibo Chen added a comment - Commented on the wrong jira?
        Hide
        rkanter Robert Kanter added a comment -

        It overall looks good to me. But is there anyway to reduce the amount of duplicate code? The handleMapContainerRequest and handleReduceContainerRequest methods are very similar.

        Show
        rkanter Robert Kanter added a comment - It overall looks good to me. But is there anyway to reduce the amount of duplicate code? The handleMapContainerRequest and handleReduceContainerRequest methods are very similar.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 18s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 6m 48s trunk passed
        +1 compile 0m 18s trunk passed
        +1 checkstyle 0m 13s trunk passed
        +1 mvnsite 0m 22s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 0m 27s trunk passed
        +1 javadoc 0m 13s trunk passed
        +1 mvninstall 0m 16s the patch passed
        +1 compile 0m 14s the patch passed
        +1 javac 0m 14s the patch passed
        +1 checkstyle 0m 10s the patch passed
        +1 mvnsite 0m 19s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 33s the patch passed
        +1 javadoc 0m 10s the patch passed
        +1 unit 3m 5s hadoop-mapreduce-client-hs in the patch passed.
        +1 asflicense 0m 15s The patch does not generate ASF License warnings.
        14m 42s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830233/mapreduce6718.003.patch
        JIRA Issue MAPREDUCE-6718
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 6c8a1855225e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 5707f88
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6738/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6738/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 48s trunk passed +1 compile 0m 18s trunk passed +1 checkstyle 0m 13s trunk passed +1 mvnsite 0m 22s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 27s trunk passed +1 javadoc 0m 13s trunk passed +1 mvninstall 0m 16s the patch passed +1 compile 0m 14s the patch passed +1 javac 0m 14s the patch passed +1 checkstyle 0m 10s the patch passed +1 mvnsite 0m 19s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 33s the patch passed +1 javadoc 0m 10s the patch passed +1 unit 3m 5s hadoop-mapreduce-client-hs in the patch passed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 14m 42s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830233/mapreduce6718.003.patch JIRA Issue MAPREDUCE-6718 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 6c8a1855225e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 5707f88 Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6738/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6738/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        haibochen Haibo Chen added a comment -

        Uploading a new patch to potentially address the issues. Let me know if more changes are needed Robert Kanter

        Show
        haibochen Haibo Chen added a comment - Uploading a new patch to potentially address the issues. Let me know if more changes are needed Robert Kanter
        Hide
        haibochen Haibo Chen added a comment -

        Thanks for your reviews, Robert Kanter.

        Something seems to be wrong with the math. When I had 0 jobs...

        Sorry for the confusing phase. I guess a better way to put it is "x% of cache is occupied." I believe the math is fine there. In the first case where you had 0 jobs, no jobs were loaded into the cache, so when the initialization was completed, 0% of the cache was occupied. In the other case, the cache was never more than 5% occupied, and the 20 jobs all-together just occupied 0.125% of the total cache. The reason why you did not see the log message within the for-loop is that the jobs (0, or 20) were too small.

        I also saw that the timing is not as useful. The idea is to print this out while the JHS is loading files and appears to be stuck

        The 10s gap you saw was taken before JHS started to initialize the job cache. Specially, it was in the process of JHS trying to create directories in HDFS (see createHistoryDirs(), there is a Thread.sleep()). I will add a if statement in the createHistoryDirs method to avoid the unnecessary 10s sleep when the operation is successful.

        Show
        haibochen Haibo Chen added a comment - Thanks for your reviews, Robert Kanter . Something seems to be wrong with the math. When I had 0 jobs... Sorry for the confusing phase. I guess a better way to put it is "x% of cache is occupied." I believe the math is fine there. In the first case where you had 0 jobs, no jobs were loaded into the cache, so when the initialization was completed, 0% of the cache was occupied. In the other case, the cache was never more than 5% occupied, and the 20 jobs all-together just occupied 0.125% of the total cache. The reason why you did not see the log message within the for-loop is that the jobs (0, or 20) were too small. I also saw that the timing is not as useful. The idea is to print this out while the JHS is loading files and appears to be stuck The 10s gap you saw was taken before JHS started to initialize the job cache. Specially, it was in the process of JHS trying to create directories in HDFS (see createHistoryDirs(), there is a Thread.sleep()). I will add a if statement in the createHistoryDirs method to avoid the unnecessary 10s sleep when the operation is successful.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 13s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 6m 46s trunk passed
        +1 compile 0m 18s trunk passed
        +1 checkstyle 0m 13s trunk passed
        +1 mvnsite 0m 22s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 0m 28s trunk passed
        +1 javadoc 0m 13s trunk passed
        +1 mvninstall 0m 16s the patch passed
        +1 compile 0m 15s the patch passed
        +1 javac 0m 15s the patch passed
        +1 checkstyle 0m 11s the patch passed
        +1 mvnsite 0m 19s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 33s the patch passed
        +1 javadoc 0m 10s the patch passed
        +1 unit 6m 37s hadoop-mapreduce-client-hs in the patch passed.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        18m 11s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817549/mapreduce6718.002.patch
        JIRA Issue MAPREDUCE-6718
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 9547b03bc9b7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 40acace
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6735/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6735/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 46s trunk passed +1 compile 0m 18s trunk passed +1 checkstyle 0m 13s trunk passed +1 mvnsite 0m 22s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 28s trunk passed +1 javadoc 0m 13s trunk passed +1 mvninstall 0m 16s the patch passed +1 compile 0m 15s the patch passed +1 javac 0m 15s the patch passed +1 checkstyle 0m 11s the patch passed +1 mvnsite 0m 19s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 33s the patch passed +1 javadoc 0m 10s the patch passed +1 unit 6m 37s hadoop-mapreduce-client-hs in the patch passed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 18m 11s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817549/mapreduce6718.002.patch JIRA Issue MAPREDUCE-6718 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 9547b03bc9b7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 40acace Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6735/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6735/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        rkanter Robert Kanter added a comment -

        Two more things I found while actually trying this out:

        1. Something seems to be wrong with the math. When I had 0 jobs, it said this:
          2016-09-22 14:40:26,650 INFO org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager: Found 0 directories to load
          2016-09-22 14:40:26,650 INFO org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager: Existing job initialization finished. 0.0% of cache is loaded.
          

          And when I had ~20 jobs, it said this:

          2016-09-22 14:52:02,491 INFO hs.HistoryFileManager: Found 1 directories to load
          2016-09-22 14:52:02,541 INFO hs.HistoryFileManager: Existing job initialization finished. 0.125% of cache is loaded.
          
        2. I also saw that the timing is not as useful. The idea is to print this out while the JHS is loading files and appears to be stuck, but when I had ~20 jobs, it had this:
          2016-09-22 14:51:52,303 INFO jobhistory.JobHistoryUtils: Default file system [hdfs://0.0.0.0:23010]
          2016-09-22 14:52:02,473 INFO hs.HistoryFileManager: Initializing Existing Jobs...
          2016-09-22 14:52:02,491 INFO hs.HistoryFileManager: Found 1 directories to load
          2016-09-22 14:52:02,541 INFO hs.HistoryFileManager: Existing job initialization finished. 0.125% of cache is loaded.
          

          Even though it was only ~10 seconds in this case, that's where the gap is. So while it starts to load, all I see in the log is the "Default file system" message, and not the "Initializing Existing Jobs..." message.

        Show
        rkanter Robert Kanter added a comment - Two more things I found while actually trying this out: Something seems to be wrong with the math. When I had 0 jobs, it said this: 2016-09-22 14:40:26,650 INFO org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager: Found 0 directories to load 2016-09-22 14:40:26,650 INFO org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager: Existing job initialization finished. 0.0% of cache is loaded. And when I had ~20 jobs, it said this: 2016-09-22 14:52:02,491 INFO hs.HistoryFileManager: Found 1 directories to load 2016-09-22 14:52:02,541 INFO hs.HistoryFileManager: Existing job initialization finished. 0.125% of cache is loaded. I also saw that the timing is not as useful. The idea is to print this out while the JHS is loading files and appears to be stuck, but when I had ~20 jobs, it had this: 2016-09-22 14:51:52,303 INFO jobhistory.JobHistoryUtils: Default file system [hdfs://0.0.0.0:23010] 2016-09-22 14:52:02,473 INFO hs.HistoryFileManager: Initializing Existing Jobs... 2016-09-22 14:52:02,491 INFO hs.HistoryFileManager: Found 1 directories to load 2016-09-22 14:52:02,541 INFO hs.HistoryFileManager: Existing job initialization finished. 0.125% of cache is loaded. Even though it was only ~10 seconds in this case, that's where the gap is. So while it starts to load, all I see in the log is the "Default file system" message, and not the "Initializing Existing Jobs..." message.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 37s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 7m 16s trunk passed
        +1 compile 0m 19s trunk passed
        +1 checkstyle 0m 14s trunk passed
        +1 mvnsite 0m 23s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 0m 29s trunk passed
        +1 javadoc 0m 14s trunk passed
        +1 mvninstall 0m 17s the patch passed
        +1 compile 0m 15s the patch passed
        +1 javac 0m 15s the patch passed
        +1 checkstyle 0m 10s the patch passed
        +1 mvnsite 0m 20s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 35s the patch passed
        +1 javadoc 0m 11s the patch passed
        +1 unit 6m 13s hadoop-mapreduce-client-hs in the patch passed.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        18m 49s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817549/mapreduce6718.002.patch
        JIRA Issue MAPREDUCE-6718
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 866cbb9ad56d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / ad256be
        Default Java 1.8.0_91
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6610/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6610/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 37s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 16s trunk passed +1 compile 0m 19s trunk passed +1 checkstyle 0m 14s trunk passed +1 mvnsite 0m 23s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 29s trunk passed +1 javadoc 0m 14s trunk passed +1 mvninstall 0m 17s the patch passed +1 compile 0m 15s the patch passed +1 javac 0m 15s the patch passed +1 checkstyle 0m 10s the patch passed +1 mvnsite 0m 20s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 35s the patch passed +1 javadoc 0m 11s the patch passed +1 unit 6m 13s hadoop-mapreduce-client-hs in the patch passed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 18m 49s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817549/mapreduce6718.002.patch JIRA Issue MAPREDUCE-6718 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 866cbb9ad56d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ad256be Default Java 1.8.0_91 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6610/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6610/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        haibochen Haibo Chen added a comment -

        Nice catch! I added a check of maxSize for "double loadedPercent = prevCacheSize * 100.0 / jobListCache.maxSize". jobListCache.isFull() checks the size against maxSize, and returns true if maxSize is zero, so we won't come to (currCacheSize - prevCacheSize)/(double)jobListCache.maxSize in the body of the for loop.

        Show
        haibochen Haibo Chen added a comment - Nice catch! I added a check of maxSize for "double loadedPercent = prevCacheSize * 100.0 / jobListCache.maxSize". jobListCache.isFull() checks the size against maxSize, and returns true if maxSize is zero, so we won't come to (currCacheSize - prevCacheSize)/(double)jobListCache.maxSize in the body of the for loop.
        Hide
        rkanter Robert Kanter added a comment -

        One thing:

        • If jobListCache.maxSize, which is loaded from a config property, is 0, then (currCacheSize - prevCacheSize)/(double)jobListCache.maxSize will be a problem.
        Show
        rkanter Robert Kanter added a comment - One thing: If jobListCache.maxSize , which is loaded from a config property, is 0 , then (currCacheSize - prevCacheSize)/(double)jobListCache.maxSize will be a problem.
        Hide
        wilfreds Wilfred Spiegelenburg added a comment -

        Patch looks good +1 for me
        indeed just logging added no new test would be needed for that.

        Show
        wilfreds Wilfred Spiegelenburg added a comment - Patch looks good +1 for me indeed just logging added no new test would be needed for that.
        Hide
        haibochen Haibo Chen added a comment -

        The patch only adds logging, so a new test may not be needed.

        Show
        haibochen Haibo Chen added a comment - The patch only adds logging, so a new test may not be needed.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 18s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 8m 1s trunk passed
        +1 compile 0m 19s trunk passed
        +1 checkstyle 0m 14s trunk passed
        +1 mvnsite 0m 26s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 0m 32s trunk passed
        +1 javadoc 0m 14s trunk passed
        +1 mvninstall 0m 20s the patch passed
        +1 compile 0m 18s the patch passed
        +1 javac 0m 18s the patch passed
        +1 checkstyle 0m 12s the patch passed
        +1 mvnsite 0m 26s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 41s the patch passed
        +1 javadoc 0m 13s the patch passed
        +1 unit 6m 25s hadoop-mapreduce-client-hs in the patch passed.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        19m 59s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:e2f6409
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12812057/mapreduce6718.001.patch
        JIRA Issue MAPREDUCE-6718
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux df7b2131f455 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 7adc4d1
        Default Java 1.8.0_91
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6566/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6566/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 8m 1s trunk passed +1 compile 0m 19s trunk passed +1 checkstyle 0m 14s trunk passed +1 mvnsite 0m 26s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 0m 32s trunk passed +1 javadoc 0m 14s trunk passed +1 mvninstall 0m 20s the patch passed +1 compile 0m 18s the patch passed +1 javac 0m 18s the patch passed +1 checkstyle 0m 12s the patch passed +1 mvnsite 0m 26s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 41s the patch passed +1 javadoc 0m 13s the patch passed +1 unit 6m 25s hadoop-mapreduce-client-hs in the patch passed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 19m 59s Subsystem Report/Notes Docker Image:yetus/hadoop:e2f6409 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12812057/mapreduce6718.001.patch JIRA Issue MAPREDUCE-6718 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux df7b2131f455 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 7adc4d1 Default Java 1.8.0_91 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6566/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6566/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        haibochen Haibo Chen added a comment -

        The patch adds conditional log to show progress during JHS startup.

        Show
        haibochen Haibo Chen added a comment - The patch adds conditional log to show progress during JHS startup.
        Hide
        haibochen Haibo Chen added a comment -

        Thanks, Wilfred Spiegelenburg. Your argument makes a lot of sense. Will reopen it and work on the patch.

        Show
        haibochen Haibo Chen added a comment - Thanks, Wilfred Spiegelenburg . Your argument makes a lot of sense. Will reopen it and work on the patch.
        Hide
        wilfreds Wilfred Spiegelenburg added a comment -

        We still should have a progress report anything more than a couple of seconds could already cause a customer to say the server has not started.

        What would happen if I have a cache in the history server setup for 150K jobs or more to be kept? Limiting the cache is OK and we already do that but customers increase the cache size because anything not in the cache can not be accessed. If they run 20K jobs a day and want 7 days to be accessible then the cache must be 150K.
        Purge of the history is set to 7 days by default which could easily do this.

        Not being able to find a history that is not in the cache is another issue which is far more difficult to fix.

        Show
        wilfreds Wilfred Spiegelenburg added a comment - We still should have a progress report anything more than a couple of seconds could already cause a customer to say the server has not started. What would happen if I have a cache in the history server setup for 150K jobs or more to be kept? Limiting the cache is OK and we already do that but customers increase the cache size because anything not in the cache can not be accessed. If they run 20K jobs a day and want 7 days to be accessible then the cache must be 150K. Purge of the history is set to 7 days by default which could easily do this. Not being able to find a history that is not in the cache is another issue which is far more difficult to fix.
        Hide
        haibochen Haibo Chen added a comment -

        Turns out MAPREDUCE-6059 limits the number of files to load in the cache. We will not see the long starting time as a result.

        Show
        haibochen Haibo Chen added a comment - Turns out MAPREDUCE-6059 limits the number of files to load in the cache. We will not see the long starting time as a result.

          People

          • Assignee:
            haibochen Haibo Chen
            Reporter:
            haibochen Haibo Chen
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development