Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4842

Shuffle race can hang reducer

VotersStop watchingWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.0.2-alpha, 0.23.5
    • Fix Version/s: 2.0.3-alpha, 0.23.6
    • Component/s: mrv2
    • Labels:
      None

      Description

      Saw an instance where the shuffle caused multiple reducers in a job to hang. It looked similar to the problem described in MAPREDUCE-3721, where the fetchers were all being told to WAIT by the MergeManager but no merge was taking place.

        Attachments

        1. MAPREDUCE-4842-2.patch
          16 kB
          Jason Darrell Lowe
        2. MAPREDUCE-4842.patch
          6 kB
          Arun Murthy
        3. MAPREDUCE-4842.patch
          14 kB
          Jason Darrell Lowe
        4. MAPREDUCE-4842.patch
          15 kB
          Arun Murthy
        5. MAPREDUCE-4842.patch
          15 kB
          Jason Darrell Lowe
        6. mapreduce-4842.patch
          6 kB
          Mariappan Asokan
        7. mapreduce-4842.patch
          6 kB
          Mariappan Asokan
        8. mapreduce-4842.patch
          6 kB
          Mariappan Asokan
        9. mapreduce-4842.patch
          15 kB
          Mariappan Asokan
        10. mapreduce-4842.patch
          15 kB
          Mariappan Asokan
        11. mapreduce-4842.patch
          15 kB
          Mariappan Asokan

        Issue Links

          Activity

            People

            • Assignee:
              masokan Mariappan Asokan
              Reporter:
              jlowe Jason Darrell Lowe

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment