Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6797

Job history server scans can become blocked on a single, slow entry

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.4.0, 2.8.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: jobhistoryserver
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      There is one more piece of code in HistoryFileManager where Synchronized keyword on HistoryFileInfo need to be removed. The JobHistoryServer contention issue is hit on our environment where stacktrace (attached) shows the HistoryFileManager$JobListCache.addIfAbsent unnecessarily waiting to lock on HistoryFileInfo.

      Synchronized on isMovePending and didMoveFail has been removed by Mapreduce-6684.

      HistoryFileInfo firstValue = cache.get(key);
          synchronized(firstValue) {  ---------------> Synchronized is not needed here
                    if (firstValue.isMovePending()) {
                      if(firstValue.didMoveFail() && 
                          firstValue.jobIndexInfo.getFinishTime() <= cutoff) {
                        cache.remove(key);
                        //Now lets try to delete it
                        try {
                          firstValue.delete();
                        } catch (IOException e) {
                          LOG.error("Error while trying to delete history files" +
                          " that could not be moved to done.", e);
                        }
                      } else {
                        LOG.warn("Waiting to remove " + key
                            + " from JobListCache because it is not in done yet.");
                      }
                    } else {
                      cache.remove(key);
                    }
                  }
      
      
      
      Note: stacktrace is from hadoop-2.4.0 version and the problem exists in latest hadoop as well
      
      "2144820863@qtp-313351300-38156" daemon prio=10 tid=0x0000000001e13800 nid=0xf133 waiting for monitor entry [0x00007f7c1d8dd000]
         java.lang.Thread.State: BLOCKED (on object monitor)
              at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager$JobListCache.addIfAbsent(HistoryFileManager.java:226)
              - waiting to lock <0x000000040145c4d8> (a org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager$HistoryFileInfo)
              at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.scanIntermediateDirectory(HistoryFileManager.java:825)
              at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.access$200(HistoryFileManager.java:82)
              at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager$UserLogDir.scanIfNeeded(HistoryFileManager.java:280)
              - locked <0x0000000400375388> (a org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager$UserLogDir)
              at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.scanIntermediateDirectory(HistoryFileManager.java:792)
              at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.getAllFileInfo(HistoryFileManager.java:920)
              at org.apache.hadoop.mapreduce.v2.hs.CachedHistoryStorage.getAllPartialJobs(CachedHistoryStorage.java:156)
              at org.apache.hadoop.mapreduce.v2.hs.JobHistory.getAllJobs(JobHistory.java:235)
      
      1. 0001-MAPREDUCE-6797.patch
        2 kB
        Prabhu Joseph
      2. 0002-MAPREDUCE-6797.patch
        3 kB
        Prabhu Joseph
      3. 0003-MAPREDUCE-6797.patch
        3 kB
        Prabhu Joseph
      4. 0004-MAPREDUCE-6797.patch
        3 kB
        Prabhu Joseph
      5. jstack
        1.12 MB
        Prabhu Joseph

        Issue Links

          Activity

          Hide
          sunilg Sunil G added a comment -

          Thanks Prabhu Joseph for raising this jira.

          under this block synchronized(firstValue) , we are majorly invoking firstValue.delete() which is having a lock and cache.remove(key). Here cache is concurrent skiplist and its safe to use.. So i think we can remove the synchronized block as mentioned. From new thread dump, which is similar to the one from MAPREDUCE-6684, it looks very clear that addIfAbsent is now the bottleneck.

          Thoughts Jason Lowe / Haibo Chen Karthik Kambatla.

          Show
          sunilg Sunil G added a comment - Thanks Prabhu Joseph for raising this jira. under this block synchronized(firstValue) , we are majorly invoking firstValue.delete() which is having a lock and cache.remove(key) . Here cache is concurrent skiplist and its safe to use.. So i think we can remove the synchronized block as mentioned. From new thread dump, which is similar to the one from MAPREDUCE-6684 , it looks very clear that addIfAbsent is now the bottleneck. Thoughts Jason Lowe / Haibo Chen Karthik Kambatla .
          Hide
          jlowe Jason Lowe added a comment -

          It looks like the lock here can be removed.

          Show
          jlowe Jason Lowe added a comment - It looks like the lock here can be removed.
          Hide
          kasha Karthik Kambatla added a comment -

          If multiple threads call addIfAbsent simultaneously, is it possible they process the same HistoryFileInfo? How do we ensure only one thread is processing a file?

          Show
          kasha Karthik Kambatla added a comment - If multiple threads call addIfAbsent simultaneously, is it possible they process the same HistoryFileInfo ? How do we ensure only one thread is processing a file?
          Hide
          Prabhu Joseph Prabhu Joseph added a comment - - edited

          Karthik Kambatla Multiple threads calling addIfAbsent simultaneously and processing a same HistoryFileInfo is possible and that won't face any issue even after removing synchronized block as the operations inside are thread safe and also idempotent.

          Show
          Prabhu Joseph Prabhu Joseph added a comment - - edited Karthik Kambatla Multiple threads calling addIfAbsent simultaneously and processing a same HistoryFileInfo is possible and that won't face any issue even after removing synchronized block as the operations inside are thread safe and also idempotent.
          Hide
          Prabhu Joseph Prabhu Joseph added a comment -

          Jason Lowe Karthik Kambatla Sunil G Need your help in reviewing the patch.

          Show
          Prabhu Joseph Prabhu Joseph added a comment - Jason Lowe Karthik Kambatla Sunil G Need your help in reviewing the patch.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 6m 49s trunk passed
          +1 compile 0m 18s trunk passed
          +1 checkstyle 0m 13s trunk passed
          +1 mvnsite 0m 22s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 0m 29s trunk passed
          +1 javadoc 0m 12s trunk passed
          +1 mvninstall 0m 17s the patch passed
          +1 compile 0m 17s the patch passed
          +1 javac 0m 17s the patch passed
          -1 checkstyle 0m 11s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs: The patch generated 24 new + 16 unchanged - 0 fixed = 40 total (was 16)
          +1 mvnsite 0m 21s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 35s the patch passed
          +1 javadoc 0m 11s the patch passed
          +1 unit 3m 6s hadoop-mapreduce-client-hs in the patch passed.
          +1 asflicense 0m 15s The patch does not generate ASF License warnings.
          14m 52s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834613/0001-MAPREDUCE-6797.patch
          JIRA Issue MAPREDUCE-6797
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 1b8bf2177109 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 1b79c41
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6783/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6783/testReport/
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6783/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 49s trunk passed +1 compile 0m 18s trunk passed +1 checkstyle 0m 13s trunk passed +1 mvnsite 0m 22s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 29s trunk passed +1 javadoc 0m 12s trunk passed +1 mvninstall 0m 17s the patch passed +1 compile 0m 17s the patch passed +1 javac 0m 17s the patch passed -1 checkstyle 0m 11s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs: The patch generated 24 new + 16 unchanged - 0 fixed = 40 total (was 16) +1 mvnsite 0m 21s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 35s the patch passed +1 javadoc 0m 11s the patch passed +1 unit 3m 6s hadoop-mapreduce-client-hs in the patch passed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 14m 52s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834613/0001-MAPREDUCE-6797.patch JIRA Issue MAPREDUCE-6797 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 1b8bf2177109 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 1b79c41 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6783/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6783/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6783/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          jlowe Jason Lowe added a comment -

          Thanks for the patch!

          As I mentioned above, I think we're OK with the lock removal, but I'd like to hear from Karthik Kambatla before committing. It'd also be good to cleanup the indentation warnings from checkstyle.

          Show
          jlowe Jason Lowe added a comment - Thanks for the patch! As I mentioned above, I think we're OK with the lock removal, but I'd like to hear from Karthik Kambatla before committing. It'd also be good to cleanup the indentation warnings from checkstyle.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 6m 48s trunk passed
          +1 compile 0m 17s trunk passed
          +1 checkstyle 0m 13s trunk passed
          +1 mvnsite 0m 23s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 0m 28s trunk passed
          +1 javadoc 0m 13s trunk passed
          +1 mvninstall 0m 17s the patch passed
          +1 compile 0m 16s the patch passed
          +1 javac 0m 16s the patch passed
          -1 checkstyle 0m 12s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs: The patch generated 2 new + 14 unchanged - 2 fixed = 16 total (was 16)
          +1 mvnsite 0m 20s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 33s the patch passed
          +1 javadoc 0m 11s the patch passed
          +1 unit 3m 16s hadoop-mapreduce-client-hs in the patch passed.
          +1 asflicense 0m 16s The patch does not generate ASF License warnings.
          15m 0s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:e809691
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12837758/0002-MAPREDUCE-6797.patch
          JIRA Issue MAPREDUCE-6797
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux f89948e428dd 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / b970446
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6801/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6801/testReport/
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6801/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 48s trunk passed +1 compile 0m 17s trunk passed +1 checkstyle 0m 13s trunk passed +1 mvnsite 0m 23s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 28s trunk passed +1 javadoc 0m 13s trunk passed +1 mvninstall 0m 17s the patch passed +1 compile 0m 16s the patch passed +1 javac 0m 16s the patch passed -1 checkstyle 0m 12s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs: The patch generated 2 new + 14 unchanged - 2 fixed = 16 total (was 16) +1 mvnsite 0m 20s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 33s the patch passed +1 javadoc 0m 11s the patch passed +1 unit 3m 16s hadoop-mapreduce-client-hs in the patch passed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 15m 0s Subsystem Report/Notes Docker Image:yetus/hadoop:e809691 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12837758/0002-MAPREDUCE-6797.patch JIRA Issue MAPREDUCE-6797 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux f89948e428dd 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b970446 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6801/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6801/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6801/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 21s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 9m 1s trunk passed
          +1 compile 0m 23s trunk passed
          +1 checkstyle 0m 17s trunk passed
          +1 mvnsite 0m 30s trunk passed
          +1 mvneclipse 0m 16s trunk passed
          +1 findbugs 0m 39s trunk passed
          +1 javadoc 0m 15s trunk passed
          +1 mvninstall 0m 22s the patch passed
          +1 compile 0m 22s the patch passed
          +1 javac 0m 22s the patch passed
          -1 checkstyle 0m 14s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs: The patch generated 2 new + 14 unchanged - 2 fixed = 16 total (was 16)
          +1 mvnsite 0m 26s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 45s the patch passed
          +1 javadoc 0m 13s the patch passed
          +1 unit 3m 45s hadoop-mapreduce-client-hs in the patch passed.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          19m 12s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:e809691
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12837779/0003-MAPREDUCE-6797.patch
          JIRA Issue MAPREDUCE-6797
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux d1b152a7b604 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / f768955
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6802/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6802/testReport/
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6802/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 9m 1s trunk passed +1 compile 0m 23s trunk passed +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 30s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 0m 39s trunk passed +1 javadoc 0m 15s trunk passed +1 mvninstall 0m 22s the patch passed +1 compile 0m 22s the patch passed +1 javac 0m 22s the patch passed -1 checkstyle 0m 14s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs: The patch generated 2 new + 14 unchanged - 2 fixed = 16 total (was 16) +1 mvnsite 0m 26s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 45s the patch passed +1 javadoc 0m 13s the patch passed +1 unit 3m 45s hadoop-mapreduce-client-hs in the patch passed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 19m 12s Subsystem Report/Notes Docker Image:yetus/hadoop:e809691 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12837779/0003-MAPREDUCE-6797.patch JIRA Issue MAPREDUCE-6797 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux d1b152a7b604 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f768955 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6802/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6802/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6802/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 7m 15s trunk passed
          +1 compile 0m 19s trunk passed
          +1 checkstyle 0m 16s trunk passed
          +1 mvnsite 0m 25s trunk passed
          +1 mvneclipse 0m 16s trunk passed
          +1 findbugs 0m 31s trunk passed
          +1 javadoc 0m 15s trunk passed
          +1 mvninstall 0m 19s the patch passed
          +1 compile 0m 17s the patch passed
          +1 javac 0m 17s the patch passed
          +1 checkstyle 0m 13s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs: The patch generated 0 new + 14 unchanged - 2 fixed = 14 total (was 16)
          +1 mvnsite 0m 22s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 38s the patch passed
          +1 javadoc 0m 11s the patch passed
          +1 unit 3m 13s hadoop-mapreduce-client-hs in the patch passed.
          +1 asflicense 0m 14s The patch does not generate ASF License warnings.
          15m 51s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:e809691
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12838018/0004-MAPREDUCE-6797.patch
          JIRA Issue MAPREDUCE-6797
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 7f2adaed76e0 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / dbb133c
          Default Java 1.8.0_101
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6803/testReport/
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6803/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 15s trunk passed +1 compile 0m 19s trunk passed +1 checkstyle 0m 16s trunk passed +1 mvnsite 0m 25s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 0m 31s trunk passed +1 javadoc 0m 15s trunk passed +1 mvninstall 0m 19s the patch passed +1 compile 0m 17s the patch passed +1 javac 0m 17s the patch passed +1 checkstyle 0m 13s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs: The patch generated 0 new + 14 unchanged - 2 fixed = 14 total (was 16) +1 mvnsite 0m 22s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 38s the patch passed +1 javadoc 0m 11s the patch passed +1 unit 3m 13s hadoop-mapreduce-client-hs in the patch passed. +1 asflicense 0m 14s The patch does not generate ASF License warnings. 15m 51s Subsystem Report/Notes Docker Image:yetus/hadoop:e809691 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12838018/0004-MAPREDUCE-6797.patch JIRA Issue MAPREDUCE-6797 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 7f2adaed76e0 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / dbb133c Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6803/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6803/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          Prabhu Joseph Prabhu Joseph added a comment -

          Thanks Jason Lowe for the review. Took some time to learn the checkstyle issues. Hi Karthik Kambatla, need your help to review this.

          Show
          Prabhu Joseph Prabhu Joseph added a comment - Thanks Jason Lowe for the review. Took some time to learn the checkstyle issues. Hi Karthik Kambatla , need your help to review this.
          Hide
          kasha Karthik Kambatla added a comment -

          I am a little too caught up for the next two weeks to take a good look. I am comfortable with the change since Jason Lowe has approved it.

          Haibo Chen - as the last person who worked on this code, are you able to verify the change?

          Show
          kasha Karthik Kambatla added a comment - I am a little too caught up for the next two weeks to take a good look. I am comfortable with the change since Jason Lowe has approved it. Haibo Chen - as the last person who worked on this code, are you able to verify the change?
          Hide
          haibochen Haibo Chen added a comment -

          Agree with Prabhu Joseph that the removal of synchronize block is safe.

          Show
          haibochen Haibo Chen added a comment - Agree with Prabhu Joseph that the removal of synchronize block is safe.
          Hide
          jlowe Jason Lowe added a comment -

          +1 for the latest patch. Committing this.

          Show
          jlowe Jason Lowe added a comment - +1 for the latest patch. Committing this.
          Hide
          jlowe Jason Lowe added a comment -

          Thanks to Prabhu Joseph for the contribution and to Sunil G, Karthik Kambatla, and Haibo Chen for additional review! I committed this to trunk, branch-2, and branch-2.8.

          Show
          jlowe Jason Lowe added a comment - Thanks to Prabhu Joseph for the contribution and to Sunil G , Karthik Kambatla , and Haibo Chen for additional review! I committed this to trunk, branch-2, and branch-2.8.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10833 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10833/)
          MAPREDUCE-6797. Job history server scans can become blocked on a single, (jlowe: rev 99c2bbd337942e4bc7b246a88dff53f98e530651)

          • (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryFileManager.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10833 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10833/ ) MAPREDUCE-6797 . Job history server scans can become blocked on a single, (jlowe: rev 99c2bbd337942e4bc7b246a88dff53f98e530651) (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryFileManager.java

            People

            • Assignee:
              Prabhu Joseph Prabhu Joseph
              Reporter:
              Prabhu Joseph Prabhu Joseph
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development