Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4315

jobhistory.jsp throws 500 when a .txt file is found in /done

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.2
    • Fix Version/s: 1.2.0
    • Component/s: jobhistoryserver
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      if a .txt file located in /done the parser throws an 500 error.
      Trace:
      java.lang.ArrayIndexOutOfBoundsException: 1
      at org.apache.hadoop.mapred.jobhistory_jsp$2.compare(jobhistory_jsp.java:295)
      at org.apache.hadoop.mapred.jobhistory_jsp$2.compare(jobhistory_jsp.java:279)
      at java.util.Arrays.mergeSort(Arrays.java:1270)
      at java.util.Arrays.mergeSort(Arrays.java:1282)
      at java.util.Arrays.mergeSort(Arrays.java:1282)
      at java.util.Arrays.mergeSort(Arrays.java:1282)
      at java.util.Arrays.mergeSort(Arrays.java:1281)
      at java.util.Arrays.mergeSort(Arrays.java:1281)
      at java.util.Arrays.mergeSort(Arrays.java:1281)
      at java.util.Arrays.mergeSort(Arrays.java:1281)
      at java.util.Arrays.mergeSort(Arrays.java:1281)
      at java.util.Arrays.mergeSort(Arrays.java:1282)
      at java.util.Arrays.mergeSort(Arrays.java:1281)
      at java.util.Arrays.sort(Arrays.java:1210)
      at org.apache.hadoop.mapred.jobhistory_jsp._jspService(jobhistory_jsp.java:279)
      at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
      at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
      at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
      at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:864)
      at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
      at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
      at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
      at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
      at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
      at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
      at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
      at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
      at org.mortbay.jetty.Server.handle(Server.java:326)
      at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
      at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
      at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
      at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
      at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
      at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
      at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

      Reproduce:

      cd ../done
      touch test.txt
      reload jobhistory

      1. MAPREDUCE-4315-2.patch
        1 kB
        Sandy Ryza
      2. MAPREDUCE-4315-1.patch
        1 kB
        Sandy Ryza
      3. MAPREDUCE-4315.patch
        2 kB
        Sandy Ryza
      4. MAPREDUCE-4315.patch
        2 kB
        Sandy Ryza

        Activity

        Alexander Alten-Lorenz created issue -
        Sandy Ryza made changes -
        Field Original Value New Value
        Attachment MAPREDUCE-4315.patch [ 12544563 ]
        Sandy Ryza made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Sandy Ryza added a comment -

        The problem occurs not because the file ends with ".txt", but because it doesn't have a prefix like "job_201209101405_0001_1347311220402".

        Show
        Sandy Ryza added a comment - The problem occurs not because the file ends with ".txt", but because it doesn't have a prefix like "job_201209101405_0001_1347311220402".
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12544563/MAPREDUCE-4315.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2841//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12544563/MAPREDUCE-4315.patch against trunk revision . -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2841//console This message is automatically generated.
        Sandy Ryza made changes -
        Attachment MAPREDUCE-4315.patch [ 12564004 ]
        Sandy Ryza made changes -
        Assignee Sandy Ryza [ sandyr ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12564004/MAPREDUCE-4315.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3220//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12564004/MAPREDUCE-4315.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3220//console This message is automatically generated.
        Hide
        Alejandro Abdelnur added a comment -

        Sandy, why not use a filter when reading files from the done/ dir to pick up only files with the right pattern? then you don't have to overload the comparator logic and you don't have to break the loop when the file does not match the pattern.

        Show
        Alejandro Abdelnur added a comment - Sandy, why not use a filter when reading files from the done/ dir to pick up only files with the right pattern? then you don't have to overload the comparator logic and you don't have to break the loop when the file does not match the pattern.
        Sandy Ryza made changes -
        Attachment MAPREDUCE-4315-1.patch [ 12565009 ]
        Hide
        Sandy Ryza added a comment -

        Uploaded a new patch that uses a filter. Verified that it works on a pseudo-distributed cluster.

        Show
        Sandy Ryza added a comment - Uploaded a new patch that uses a filter. Verified that it works on a pseudo-distributed cluster.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12565009/MAPREDUCE-4315-1.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3243//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12565009/MAPREDUCE-4315-1.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3243//console This message is automatically generated.
        Sandy Ryza made changes -
        Attachment MAPREDUCE-4315-2.patch [ 12565204 ]
        Sandy Ryza made changes -
        Attachment MAPREDUCE-4315-2.patch [ 12565206 ]
        Sandy Ryza made changes -
        Attachment MAPREDUCE-4315-2.patch [ 12565204 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12565206/MAPREDUCE-4315-2.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3246//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12565206/MAPREDUCE-4315-2.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3246//console This message is automatically generated.
        Hide
        Sandy Ryza added a comment -

        Tested with pseudo-distributed cluster and it works with the latest patch and doesn't work without it.

        Show
        Sandy Ryza added a comment - Tested with pseudo-distributed cluster and it works with the latest patch and doesn't work without it.
        Hide
        Alejandro Abdelnur added a comment -

        +1

        Show
        Alejandro Abdelnur added a comment - +1
        Hide
        Alejandro Abdelnur added a comment -

        Thanks Sandy. Committed to branch-1.

        Show
        Alejandro Abdelnur added a comment - Thanks Sandy. Committed to branch-1.
        Alejandro Abdelnur made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Fix Version/s 1.2.0 [ 12321661 ]
        Resolution Fixed [ 1 ]
        Hide
        Matt Foley added a comment -

        Closed upon release of Hadoop 1.2.0.

        Show
        Matt Foley added a comment - Closed upon release of Hadoop 1.2.0.
        Matt Foley made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        97d 16h 5m 1 Sandy Ryza 11/Sep/12 01:07
        Patch Available Patch Available Resolved Resolved
        127d 23h 48m 1 Alejandro Abdelnur 16/Jan/13 23:56
        Resolved Resolved Closed Closed
        118d 5h 19m 1 Matt Foley 15/May/13 06:15

          People

          • Assignee:
            Sandy Ryza
            Reporter:
            Alexander Alten-Lorenz
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development