Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.2, 0.20.205.0
    • Fix Version/s: 0.20.205.0
    • Component/s: None
    • Labels:
      None
    • Release Note:
      I just committed this. Thanks Anupam!

      Description

      During the execution of a large job, the map tasks timeout:

      INFO mapred.JobClient: Task Id : attempt_201010290414_60974_m_000057_1, Status : FAILED
      Task attempt_201010290414_60974_m_000057_1 failed to report status for 609 seconds. Killing!
      

      The bug is in the fact that the mapper has already finished, and, according to the logs, the timeout occurs during the merge sort phase.
      The intermediate data generated by the map task is quite large. So I think this is the problem.

      The logs show that the merge-sort was running for 10 minutes when the task was killed.
      I think the mapred.Merger should call Reporter.progress() somewhere.

      1. MAPREDUCE-2187-20-security.patch
        6 kB
        Anupam Seth
      2. MAPREDUCE-2187-20-security-v2.patch
        7 kB
        Anupam Seth
      3. MAPREDUCE-2187-22.patch
        6 kB
        Anupam Seth
      4. MAPREDUCE-2187-branch-MR-279.patch
        7 kB
        Anupam Seth
      5. MAPREDUCE-2187-MR-279-v2.patch
        8 kB
        Anupam Seth
      6. MAPREDUCE-2187-trunk.patch
        6 kB
        Anupam Seth
      7. MAPREDUCE-2187-trunk-v2.patch
        7 kB
        Anupam Seth
      8. MAPREDUCE-2187-trunk-v3.patch
        8 kB
        Anupam Seth

        Issue Links

          Activity

          Matt Foley made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Arun C Murthy made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Release Note I just committed this. Thanks Anupam!
          Resolution Fixed [ 1 ]
          Anupam Seth made changes -
          Attachment MAPREDUCE-2187-trunk-v3.patch [ 12488472 ]
          Anupam Seth made changes -
          Attachment MAPREDUCE-2187-trunk-v2.patch [ 12488279 ]
          Anupam Seth made changes -
          Attachment MAPREDUCE-2187-MR-279-v2.patch [ 12488278 ]
          Anupam Seth made changes -
          Attachment MAPREDUCE-2187-20-security-v2.patch [ 12488277 ]
          Nathan Roberts made changes -
          Fix Version/s 0.20.205.0 [ 12316391 ]
          Sherry Chen made changes -
          Affects Version/s 0.20.205.0 [ 12316391 ]
          Sherry Chen made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Anupam Seth made changes -
          Attachment MAPREDUCE-2187-branch-MR-279.patch [ 12483443 ]
          Anupam Seth made changes -
          Attachment MAPREDUCE-2187-22.patch [ 12481750 ]
          Anupam Seth made changes -
          Attachment MAPREDUCE-2187-trunk.patch [ 12481748 ]
          Anupam Seth made changes -
          Attachment MAPREDUCE-2187-20-security.patch [ 12481720 ]
          Anupam Seth made changes -
          Assignee Anupam Seth [ anupamseth ]
          Luke Lu made changes -
          Field Original Value New Value
          Link This issue relates to MAPREDUCE-2177 [ MAPREDUCE-2177 ]
          Gianmarco De Francisci Morales created issue -

            People

            • Assignee:
              Anupam Seth
              Reporter:
              Gianmarco De Francisci Morales
            • Votes:
              1 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development