Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4315

jobhistory.jsp throws 500 when a .txt file is found in /done

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.2
    • Fix Version/s: 1.2.0
    • Component/s: jobhistoryserver
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      if a .txt file located in /done the parser throws an 500 error.
      Trace:
      java.lang.ArrayIndexOutOfBoundsException: 1
      at org.apache.hadoop.mapred.jobhistory_jsp$2.compare(jobhistory_jsp.java:295)
      at org.apache.hadoop.mapred.jobhistory_jsp$2.compare(jobhistory_jsp.java:279)
      at java.util.Arrays.mergeSort(Arrays.java:1270)
      at java.util.Arrays.mergeSort(Arrays.java:1282)
      at java.util.Arrays.mergeSort(Arrays.java:1282)
      at java.util.Arrays.mergeSort(Arrays.java:1282)
      at java.util.Arrays.mergeSort(Arrays.java:1281)
      at java.util.Arrays.mergeSort(Arrays.java:1281)
      at java.util.Arrays.mergeSort(Arrays.java:1281)
      at java.util.Arrays.mergeSort(Arrays.java:1281)
      at java.util.Arrays.mergeSort(Arrays.java:1281)
      at java.util.Arrays.mergeSort(Arrays.java:1282)
      at java.util.Arrays.mergeSort(Arrays.java:1281)
      at java.util.Arrays.sort(Arrays.java:1210)
      at org.apache.hadoop.mapred.jobhistory_jsp._jspService(jobhistory_jsp.java:279)
      at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
      at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
      at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
      at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:864)
      at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
      at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
      at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
      at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
      at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
      at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
      at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
      at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
      at org.mortbay.jetty.Server.handle(Server.java:326)
      at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
      at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
      at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
      at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
      at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
      at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
      at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

      Reproduce:

      cd ../done
      touch test.txt
      reload jobhistory

      1. MAPREDUCE-4315-2.patch
        1 kB
        Sandy Ryza
      2. MAPREDUCE-4315-1.patch
        1 kB
        Sandy Ryza
      3. MAPREDUCE-4315.patch
        2 kB
        Sandy Ryza
      4. MAPREDUCE-4315.patch
        2 kB
        Sandy Ryza

        Activity

        Hide
        Sandy Ryza added a comment -

        The problem occurs not because the file ends with ".txt", but because it doesn't have a prefix like "job_201209101405_0001_1347311220402".

        Show
        Sandy Ryza added a comment - The problem occurs not because the file ends with ".txt", but because it doesn't have a prefix like "job_201209101405_0001_1347311220402".
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12544563/MAPREDUCE-4315.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2841//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12544563/MAPREDUCE-4315.patch against trunk revision . -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2841//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12564004/MAPREDUCE-4315.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3220//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12564004/MAPREDUCE-4315.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3220//console This message is automatically generated.
        Hide
        Alejandro Abdelnur added a comment -

        Sandy, why not use a filter when reading files from the done/ dir to pick up only files with the right pattern? then you don't have to overload the comparator logic and you don't have to break the loop when the file does not match the pattern.

        Show
        Alejandro Abdelnur added a comment - Sandy, why not use a filter when reading files from the done/ dir to pick up only files with the right pattern? then you don't have to overload the comparator logic and you don't have to break the loop when the file does not match the pattern.
        Hide
        Sandy Ryza added a comment -

        Uploaded a new patch that uses a filter. Verified that it works on a pseudo-distributed cluster.

        Show
        Sandy Ryza added a comment - Uploaded a new patch that uses a filter. Verified that it works on a pseudo-distributed cluster.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12565009/MAPREDUCE-4315-1.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3243//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12565009/MAPREDUCE-4315-1.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3243//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12565206/MAPREDUCE-4315-2.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3246//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12565206/MAPREDUCE-4315-2.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3246//console This message is automatically generated.
        Hide
        Sandy Ryza added a comment -

        Tested with pseudo-distributed cluster and it works with the latest patch and doesn't work without it.

        Show
        Sandy Ryza added a comment - Tested with pseudo-distributed cluster and it works with the latest patch and doesn't work without it.
        Hide
        Alejandro Abdelnur added a comment -

        +1

        Show
        Alejandro Abdelnur added a comment - +1
        Hide
        Alejandro Abdelnur added a comment -

        Thanks Sandy. Committed to branch-1.

        Show
        Alejandro Abdelnur added a comment - Thanks Sandy. Committed to branch-1.
        Hide
        Matt Foley added a comment -

        Closed upon release of Hadoop 1.2.0.

        Show
        Matt Foley added a comment - Closed upon release of Hadoop 1.2.0.

          People

          • Assignee:
            Sandy Ryza
            Reporter:
            Alexander Alten-Lorenz
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development