Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4842

Shuffle race can hang reducer

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 2.0.2-alpha, 0.23.5
    • 2.0.3-alpha, 0.23.6
    • mrv2
    • None

    Description

      Saw an instance where the shuffle caused multiple reducers in a job to hang. It looked similar to the problem described in MAPREDUCE-3721, where the fetchers were all being told to WAIT by the MergeManager but no merge was taking place.

      Attachments

        1. MAPREDUCE-4842.patch
          6 kB
          Arun Murthy
        2. MAPREDUCE-4842.patch
          14 kB
          Jason Darrell Lowe
        3. MAPREDUCE-4842.patch
          15 kB
          Arun Murthy
        4. MAPREDUCE-4842.patch
          15 kB
          Jason Darrell Lowe
        5. mapreduce-4842.patch
          6 kB
          Mariappan Asokan
        6. mapreduce-4842.patch
          6 kB
          Mariappan Asokan
        7. MAPREDUCE-4842-2.patch
          16 kB
          Jason Darrell Lowe
        8. mapreduce-4842.patch
          6 kB
          Mariappan Asokan
        9. mapreduce-4842.patch
          15 kB
          Mariappan Asokan
        10. mapreduce-4842.patch
          15 kB
          Mariappan Asokan
        11. mapreduce-4842.patch
          15 kB
          Mariappan Asokan

        Issue Links

          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-5423 Rare deadlock situation when reducers try to fetch map output

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. TEZ-3293 Fetch failures can cause a shuffle hang waiting for memory merge that never starts

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-3721 Race in shuffle can cause it to hang

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Activity

            People

              masokan Mariappan Asokan
              jlowe Jason Darrell Lowe
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: