Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-3114

Shuffle OOM due to EventMetaData flood

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.7.0
    • 0.7.1, 0.8.3
    • None
    • None
    • Reviewed

    Description

      A task encountered an OOM during shuffle, and investigation of the heap dump showed a lot of memory being consumed by almost 3.5 million EventMetaData objects. Auto-parallelism had reduced the number of tasks in the vertex to 1 and there were 2000 upstream tasks to shuffle.

      Attachments

        1. TEZ-3114.003.patch
          8 kB
          Jason Darrell Lowe
        2. TEZ-3114.002.patch
          4 kB
          Jason Darrell Lowe
        3. TEZ-3114.001.patch
          5 kB
          Jason Darrell Lowe

        Issue Links

          blocks

          Bug - A problem which impairs or prevents the functions of the product. TEZ-3116 Umbrella ticket for shuffle fixes

          • Major - Major loss of function.
          • Open
          relates to

          Bug - A problem which impairs or prevents the functions of the product. TEZ-3014 OOM during Shuffle in JDK 8

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              jlowe Jason Darrell Lowe
              jlowe Jason Darrell Lowe
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: