Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5

Shuffle's getMapOutput() fails with EofException, followed by IllegalStateException

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Not A Problem
    • Affects Version/s: 0.20.2, 1.1.1
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Environment:

      Sun Java 1.6.0_13, OpenSolaris, running on a SunFire 4150 (x64) 10 node cluster

      Description

      During the shuffle phase, I'm seeing a large sequence of the following actions:

      1) WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_200905181452_0002_m_000010_0,0) failed : org.mortbay.jetty.EofException
      2) WARN org.mortbay.log: Committed before 410 getMapOutput(attempt_200905181452_0002_m_000010_0,0) failed : org.mortbay.jetty.EofException
      3) ERROR org.mortbay.log: /mapOutput java.lang.IllegalStateException: Committed

      The map phase completes with 100%, and then the reduce phase crawls along with the above errors in each of the TaskTracker logs. None of the tasktrackers get lost. When I run non-data jobs like the 'pi' test from the example jar, everything works fine.

      1. temp.rar
        897 kB
        xieguiming

        Issue Links

          Activity

          George Porter created issue -
          George Porter made changes -
          Field Original Value New Value
          Link This issue is related to HADOOP-861 [ HADOOP-861 ]
          Owen O'Malley made changes -
          Project Hadoop Common [ 12310240 ] Hadoop Map/Reduce [ 12310941 ]
          Key HADOOP-5865 MAPREDUCE-5
          Component/s mapred [ 12310690 ]
          Allen Wittenauer made changes -
          Affects Version/s 0.20.2 [ 12314205 ]
          xieguiming made changes -
          Assignee xieguiming [ xieguiming ]
          xieguiming made changes -
          Assignee xieguiming [ xieguiming ]
          xieguiming made changes -
          Attachment temp.rar [ 12529886 ]
          Jan Kunigk made changes -
          Affects Version/s 1.1.1 [ 12321660 ]
          Harsh J made changes -
          Link This issue is superceded by MAPREDUCE-5492 [ MAPREDUCE-5492 ]
          Harsh J made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Not A Problem [ 8 ]

            People

            • Assignee:
              Unassigned
              Reporter:
              George Porter
            • Votes:
              12 Vote for this issue
              Watchers:
              29 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development