Hadoop Common
  1. Hadoop Common
  2. HADOOP-6109

Handle large (several MB) text input lines in a reasonable amount of time

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.19.0
    • Fix Version/s: 0.21.0
    • Component/s: io
    • Labels:
      None
    • Environment:

      Linux 2.6 kernel, java 1.6 AMD Dual-Core Opteron 2.6GHz with 1M L1/L2 cache 1.8G RAM

    • Hadoop Flags:
      Reviewed

      Description

      problem:
      =======
      hadoop was timing out on a simple pass-through job (with the default 10 min timeout)

      cause:
      =====
      i hunted this down to how Text lines are being processed inside org.apache.hadoop.util.LineReader.
      i have a fix, a task that took more than 20 minutes and still failed to complete, completes with this fix in under 30 s.
      i attach the patch (for trunk)

      the problem traces:
      ================

      hadoop version: 0.19.0
      userlogs on slave node:

      2009-05-29 13:57:33,551 WARN org.apache.hadoop.mapred.TaskRunner: Parent died. Exiting attempt_200905281652_0013_m_000006_1
      [root@domU-12-31-38-01-7C-92 attempt_200905281652_0013_m_000006_1]#

      tellingly, the last input line processed right before this WARN is 19K. (i log the full input line in the map function for debugging)

      output on map-reduce task:

      Task attempt_200905281652_0013_m_000006_2 failed to report status for 600 seconds. Killing!
      09/05/29 14:08:01 INFO mapred.JobClient: map 99% reduce 32%
      09/05/29 14:18:05 INFO mapred.JobClient: map 98% reduce 32%
      java.io.IOException: Job failed!
      at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1217)
      at com.adxpose.data.mr.DailyHeatmapAggregator.run(DailyHeatmapAggregator.java:547)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
      at com.adxpose.data.mr.DailyHeatmapAggregator.main(DailyHeatmapAggregator.java:553)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.apache.hadoop.util.RunJar.main(RunJar.java:165)
      at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
      at org.apache.hadoop.mapred.JobShell.main(JobShell.java:68)

      1. HADOOP-1234.patch
        0.9 kB
        thushara wijeratna
      2. HADOOP-1234.patch
        1 kB
        thushara wijeratna

        Issue Links

          Activity

          thushara wijeratna created issue -
          thushara wijeratna made changes -
          Field Original Value New Value
          Attachment HADOOP-1234.patch [ 12411871 ]
          Attachment HADOOP-1234.patch [ 12411872 ]
          thushara wijeratna made changes -
          Attachment HADOOP-1234.patch [ 12411871 ]
          thushara wijeratna made changes -
          Attachment HADOOP-1234.patch [ 12411966 ]
          Chris Douglas made changes -
          Component/s io [ 12310687 ]
          Component/s util [ 12310740 ]
          Chris Douglas made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Assignee thushara wijeratna [ thushara ]
          Chris Douglas made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Fix Version/s 0.21.0 [ 12313563 ]
          Resolution Fixed [ 1 ]
          Tom White made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Daniel Dai made changes -
          Link This issue breaks PIG-2374 [ PIG-2374 ]
          Harsh J made changes -
          Link This issue is broken by HADOOP-7940 [ HADOOP-7940 ]
          Harsh J made changes -
          Link This issue is broken by HADOOP-7940 [ HADOOP-7940 ]

            People

            • Assignee:
              thushara wijeratna
              Reporter:
              thushara wijeratna
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development