Hadoop Common
  1. Hadoop Common
  2. HADOOP-4933

ConcurrentModificationException in JobHistory.java

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.20.0
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Fixed a synchronization bug in job history content parsing that could result in garbled history data or a ConcurrentModificationException.

      Description

      JobHistory.java throws ConcurrentModificationException while finding out the job history version.

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          51d 23h 1 Amar Kamat 13/Feb/09 05:37
          Patch Available Patch Available Resolved Resolved
          11m 5s 1 Devaraj Das 13/Feb/09 05:48
          Resolved Resolved Closed Closed
          557d 14h 45m 1 Tom White 24/Aug/10 21:34
          Gavin made changes -
          Link This issue is depended upon by MAPREDUCE-177 [ MAPREDUCE-177 ]
          Gavin made changes -
          Link This issue blocks MAPREDUCE-177 [ MAPREDUCE-177 ]
          Tom White made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Hemanth Yamijala made changes -
          Release Note Fixed a synchronization bug in job history content parsing that could result in garbled history data or a ConcurrentModificationException.
          Hide
          Koji Noguchi added a comment -

          FYI, when hitting this bug,
          1) Job history look-up can fail with error as reported on this Jira.
          or
          2) Job history look-up can show information of completely different job.

          (2) is worse. Maybe backport to 0.20 as well?

          Show
          Koji Noguchi added a comment - FYI, when hitting this bug, 1) Job history look-up can fail with error as reported on this Jira. or 2) Job history look-up can show information of completely different job. (2) is worse. Maybe backport to 0.20 as well?
          Hide
          Hemanth Yamijala added a comment -

          The existing tests related to job history verify the fix as well. Difficult to specifically test this condition, because it happens due to concurrent access that is purely timing based. I am +1 for the patch applied to the Yahoo! hadoop distribution.

          Show
          Hemanth Yamijala added a comment - The existing tests related to job history verify the fix as well. Difficult to specifically test this condition, because it happens due to concurrent access that is purely timing based. I am +1 for the patch applied to the Yahoo! hadoop distribution.
          Hide
          Amar Kamat added a comment -

          TestLoadGenerator FAILED, TestBlockReplacement FAILED (timeout), TestHdfsProxy FAILED, TestJobTrackerRestartWithCS FAILED (timeout).
          None of these seems related to the patch.

          Show
          Amar Kamat added a comment - TestLoadGenerator FAILED, TestBlockReplacement FAILED (timeout), TestHdfsProxy FAILED, TestJobTrackerRestartWithCS FAILED (timeout). None of these seems related to the patch.
          Hide
          Amar Kamat added a comment -

          This should also be Yahoo's distribution of hadoop. The uploaded patch applies. Result of test-patch
          [exec] -1 overall.
          [exec]
          [exec] +1 @author. The patch does not contain any @author tags.
          [exec]
          [exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
          [exec] Please justify why no tests are needed for this patch.
          [exec]
          [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
          [exec]
          [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
          [exec]
          [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
          [exec]
          [exec] -1 Eclipse classpath. The patch causes the Eclipse classpath to differ from the contents of the lib directories.

          The eclipse class path issues is because jackson-core-asl-1.0.1.jar and jackson-mapper-asl-1.0.1.jar info is not updated.
          Running ant tests.

          Show
          Amar Kamat added a comment - This should also be Yahoo's distribution of hadoop. The uploaded patch applies. Result of test-patch [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no tests are needed for this patch. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] -1 Eclipse classpath. The patch causes the Eclipse classpath to differ from the contents of the lib directories. The eclipse class path issues is because jackson-core-asl-1.0.1.jar and jackson-mapper-asl-1.0.1.jar info is not updated. Running ant tests.
          Owen O'Malley made changes -
          Component/s mapred [ 12310690 ]
          Hide
          Hudson added a comment -
          Show
          Hudson added a comment - Integrated in Hadoop-trunk #756 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/756/ )
          Devaraj Das made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hadoop Flags [Reviewed]
          Fix Version/s 0.21.0 [ 12313563 ]
          Fix Version/s 0.20.0 [ 12313438 ]
          Hide
          Devaraj Das added a comment -

          I just committed this. Thanks, Amar!

          Show
          Devaraj Das added a comment - I just committed this. Thanks, Amar!
          Amar Kamat made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Amar Kamat added a comment -

          Result of test-patch on my box

          [exec] -1 overall.  
               [exec] 
               [exec]     +1 @author.  The patch does not contain any @author tags.
               [exec] 
               [exec]     -1 tests included.  The patch doesn't appear to include any new or modified tests.
               [exec]                         Please justify why no tests are needed for this patch.
               [exec] 
               [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
               [exec] 
               [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
               [exec] 
               [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
               [exec] 
               [exec]     +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
          
          Show
          Amar Kamat added a comment - Result of test-patch on my box [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no tests are needed for this patch. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
          Amar Kamat made changes -
          Attachment HADOOP-4933-v1.1.patch [ 12397116 ]
          Hide
          Amar Kamat added a comment -

          A straightforward fix. Tested it manually and it works.

          Show
          Amar Kamat added a comment - A straightforward fix. Tested it manually and it works.
          Amar Kamat made changes -
          Priority Critical [ 2 ] Blocker [ 1 ]
          Amar Kamat made changes -
          Assignee Amar Kamat [ amar_kamat ]
          Amar Kamat made changes -
          Fix Version/s 0.20.0 [ 12313438 ]
          Affects Version/s 0.20.0 [ 12313438 ]
          Amar Kamat made changes -
          Field Original Value New Value
          Link This issue blocks HADOOP-4766 [ HADOOP-4766 ]
          Hide
          Amar Kamat added a comment -

          One more. Note that these errors are seen when I try to parse history files in parallel.

          java.lang.NullPointerException
          at java.util.TreeMap.getEntry(TreeMap.java:324)
          at java.util.TreeMap.get(TreeMap.java:255)
          at org.apache.hadoop.mapred.DefaultJobHistoryParser$JobTasksParseListener.getTask(DefaultJobHistoryParser.java:67)
          at org.apache.hadoop.mapred.DefaultJobHistoryParser$JobTasksParseListener.getMapAttempt(DefaultJobHistoryParser.java:79)
          at org.apache.hadoop.mapred.DefaultJobHistoryParser$JobTasksParseListener.handle(DefaultJobHistoryParser.java:119)
          at org.apache.hadoop.mapred.JobHistory.parseLine(JobHistory.java:309)
          at org.apache.hadoop.mapred.JobHistory.parseHistoryFromFS(JobHistory.java:275)
          at org.apache.hadoop.mapred.DefaultJobHistoryParser.parseJobTasks(DefaultJobHistoryParser.java:50)
          at org.apache.hadoop.mapred.loadhistory_jsp._jspService(loadhistory_jsp.java:86)
          at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
          at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
          at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
          at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:363)
          at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
          at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
          at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
          at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
          at org.mortbay.jetty.servlet.Dispatcher.include(Dispatcher.java:192)
          at org.apache.jasper.runtime.JspRuntimeLibrary.include(JspRuntimeLibrary.java:966)
          at org.apache.hadoop.mapred.jobdetailshistory_jsp._jspService(jobdetailshistory_jsp.java:54)
          at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
          at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
          at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
          at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:363)
          at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
          at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
          at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
          at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
          at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
          at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
          at org.mortbay.jetty.Server.handle(Server.java:324)
          at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
          at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864)
          at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533)
          at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207)
          at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403)
          at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
          at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)
          
          Show
          Amar Kamat added a comment - One more. Note that these errors are seen when I try to parse history files in parallel. java.lang.NullPointerException at java.util.TreeMap.getEntry(TreeMap.java:324) at java.util.TreeMap.get(TreeMap.java:255) at org.apache.hadoop.mapred.DefaultJobHistoryParser$JobTasksParseListener.getTask(DefaultJobHistoryParser.java:67) at org.apache.hadoop.mapred.DefaultJobHistoryParser$JobTasksParseListener.getMapAttempt(DefaultJobHistoryParser.java:79) at org.apache.hadoop.mapred.DefaultJobHistoryParser$JobTasksParseListener.handle(DefaultJobHistoryParser.java:119) at org.apache.hadoop.mapred.JobHistory.parseLine(JobHistory.java:309) at org.apache.hadoop.mapred.JobHistory.parseHistoryFromFS(JobHistory.java:275) at org.apache.hadoop.mapred.DefaultJobHistoryParser.parseJobTasks(DefaultJobHistoryParser.java:50) at org.apache.hadoop.mapred.loadhistory_jsp._jspService(loadhistory_jsp.java:86) at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:363) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417) at org.mortbay.jetty.servlet.Dispatcher.include(Dispatcher.java:192) at org.apache.jasper.runtime.JspRuntimeLibrary.include(JspRuntimeLibrary.java:966) at org.apache.hadoop.mapred.jobdetailshistory_jsp._jspService(jobdetailshistory_jsp.java:54) at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:363) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:324) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)
          Hide
          Amar Kamat added a comment -

          One more. This was while trying to analyse a running job.

          org.apache.hadoop.fs.ChecksumException: Checksum error: file:job-file at offset
          	at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.readChunk(ChecksumFileSystem.java:219)
          	at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:237)
          	at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:189)
          	at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158)
          	at java.io.DataInputStream.read(DataInputStream.java:132)
          	at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:264)
          	at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:306)
          	at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158)
          	at java.io.InputStreamReader.read(InputStreamReader.java:167)
          	at java.io.BufferedReader.fill(BufferedReader.java:136)
          	at java.io.BufferedReader.readLine(BufferedReader.java:299)
          	at java.io.BufferedReader.readLine(BufferedReader.java:362)
          	at org.apache.hadoop.mapred.JobHistory.parseHistoryFromFS(JobHistory.java:277)
          	at org.apache.hadoop.mapred.DefaultJobHistoryParser.parseJobTasks(DefaultJobHistoryParser.java:50)
          	at org.apache.hadoop.mapred.loadhistory_jsp._jspService(loadhistory_jsp.java:86)
          	at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
          	at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
          	at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
          	at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:363)
          	at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
          	at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
          	at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
          	at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
          	at org.mortbay.jetty.servlet.Dispatcher.include(Dispatcher.java:192)
          	at org.apache.jasper.runtime.JspRuntimeLibrary.include(JspRuntimeLibrary.java:966)
          	at org.apache.hadoop.mapred.jobdetailshistory_jsp._jspService(jobdetailshistory_jsp.java:54)
          	at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
          	at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
          	at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
          	at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:363)
          	at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
          	at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
          	at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
          	at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
          	at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
          	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
          	at org.mortbay.jetty.Server.handle(Server.java:324)
          	at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
          	at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864)
          	at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533)
          	at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207)
          	at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403)
          	at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
          	at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)
          
          Show
          Amar Kamat added a comment - One more. This was while trying to analyse a running job. org.apache.hadoop.fs.ChecksumException: Checksum error: file:job-file at offset at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.readChunk(ChecksumFileSystem.java:219) at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:237) at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:189) at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158) at java.io.DataInputStream.read(DataInputStream.java:132) at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:264) at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:306) at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158) at java.io.InputStreamReader.read(InputStreamReader.java:167) at java.io.BufferedReader.fill(BufferedReader.java:136) at java.io.BufferedReader.readLine(BufferedReader.java:299) at java.io.BufferedReader.readLine(BufferedReader.java:362) at org.apache.hadoop.mapred.JobHistory.parseHistoryFromFS(JobHistory.java:277) at org.apache.hadoop.mapred.DefaultJobHistoryParser.parseJobTasks(DefaultJobHistoryParser.java:50) at org.apache.hadoop.mapred.loadhistory_jsp._jspService(loadhistory_jsp.java:86) at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:363) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417) at org.mortbay.jetty.servlet.Dispatcher.include(Dispatcher.java:192) at org.apache.jasper.runtime.JspRuntimeLibrary.include(JspRuntimeLibrary.java:966) at org.apache.hadoop.mapred.jobdetailshistory_jsp._jspService(jobdetailshistory_jsp.java:54) at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:363) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:324) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)
          Hide
          Amar Kamat added a comment -

          Here is the stack trace

          java.util.ConcurrentModificationException
          	at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
          	at java.util.HashMap$EntryIterator.next(HashMap.java:833)
          	at java.util.HashMap$EntryIterator.next(HashMap.java:831)
          	at java.util.HashMap.putAll(HashMap.java:523)
          	at org.apache.hadoop.mapred.JobHistory$KeyValuePair.set(JobHistory.java:428)
          	at org.apache.hadoop.mapred.JobHistory$KeyValuePair.handle(JobHistory.java:435)
          	at org.apache.hadoop.mapred.JobHistory$MetaInfoManager.handle(JobHistory.java:206)
          	at org.apache.hadoop.mapred.JobHistory.parseLine(JobHistory.java:309)
          	at org.apache.hadoop.mapred.JobHistory.access$000(JobHistory.java:75)
          	at org.apache.hadoop.mapred.JobHistory$MetaInfoManager.(JobHistory.java:183)
          	at org.apache.hadoop.mapred.JobHistory.parseHistoryFromFS(JobHistory.java:261)
          	at org.apache.hadoop.mapred.jobdetailshistory_jsp._jspService(jobdetailshistory_jsp.java:399)
          	at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
          	at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
          	at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
          	at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:363)
          	at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
          	at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
          	at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
          	at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
          	at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
          	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
          	at org.mortbay.jetty.Server.handle(Server.java:324)
          	at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
          	at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864)
          	at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533)
          	at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207)
          	at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403)
          	at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
          	at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)
          
          
          
          Show
          Amar Kamat added a comment - Here is the stack trace java.util.ConcurrentModificationException at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793) at java.util.HashMap$EntryIterator.next(HashMap.java:833) at java.util.HashMap$EntryIterator.next(HashMap.java:831) at java.util.HashMap.putAll(HashMap.java:523) at org.apache.hadoop.mapred.JobHistory$KeyValuePair.set(JobHistory.java:428) at org.apache.hadoop.mapred.JobHistory$KeyValuePair.handle(JobHistory.java:435) at org.apache.hadoop.mapred.JobHistory$MetaInfoManager.handle(JobHistory.java:206) at org.apache.hadoop.mapred.JobHistory.parseLine(JobHistory.java:309) at org.apache.hadoop.mapred.JobHistory.access$000(JobHistory.java:75) at org.apache.hadoop.mapred.JobHistory$MetaInfoManager.(JobHistory.java:183) at org.apache.hadoop.mapred.JobHistory.parseHistoryFromFS(JobHistory.java:261) at org.apache.hadoop.mapred.jobdetailshistory_jsp._jspService(jobdetailshistory_jsp.java:399) at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:363) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:324) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)
          Amar Kamat created issue -

            People

            • Assignee:
              Amar Kamat
              Reporter:
              Amar Kamat
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development