Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4842

Shuffle race can hang reducer

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 2.0.2-alpha, 0.23.5
    • 2.0.3-alpha, 0.23.6
    • mrv2
    • None

    Description

      Saw an instance where the shuffle caused multiple reducers in a job to hang. It looked similar to the problem described in MAPREDUCE-3721, where the fetchers were all being told to WAIT by the MergeManager but no merge was taking place.

      Attachments

        1. mapreduce-4842.patch
          15 kB
          Mariappan Asokan
        2. mapreduce-4842.patch
          15 kB
          Mariappan Asokan
        3. mapreduce-4842.patch
          15 kB
          Mariappan Asokan
        4. mapreduce-4842.patch
          6 kB
          Mariappan Asokan
        5. mapreduce-4842.patch
          6 kB
          Mariappan Asokan
        6. mapreduce-4842.patch
          6 kB
          Mariappan Asokan
        7. MAPREDUCE-4842.patch
          15 kB
          Jason Darrell Lowe
        8. MAPREDUCE-4842.patch
          15 kB
          Arun Murthy
        9. MAPREDUCE-4842.patch
          14 kB
          Jason Darrell Lowe
        10. MAPREDUCE-4842.patch
          6 kB
          Arun Murthy
        11. MAPREDUCE-4842-2.patch
          16 kB
          Jason Darrell Lowe

        Issue Links

          Activity

            People

              masokan Mariappan Asokan
              jlowe Jason Darrell Lowe
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: