Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-1911

MergeManager's unconditionalReserve() should check for memory limits before allocating memory to IntermediateMemoryToMemoryMerger

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.7.1, 0.8.3
    • None
    • None
    • Reviewed

    Description

      Currently, IntermediateMemoryToMemoryMerger invokes unconditionalReserve() to get the memory needed for intermediate mem-to-mem merging. It could potentially cause issue in the following scenario

      1. tez.runtime.io.sort.factor set to 100 and assume shuffled data (e.g 60% memory occupied) haven't reached TEZ_RUNTIME_SHUFFLE_MERGE_PERCENT_DEFAULT
      2. Assume that it reaches the sort.factor threshold before reaching merge threshold. This would kick in IntermediateMemoryToMemoryMerger.

      In IntermediateMemoryToMemoryMerger, it would try to allocate additional 60% without any boundary checks. This could lead to OOM depending on the tez.runtime.io.sort.mb setting.

      Attachments

        1. TEZ-1911.addendum.findbugs.patch
          2 kB
          Rajesh Balamohan
        2. TEZ-1911.3.patch
          15 kB
          Rajesh Balamohan
        3. TEZ-1911.2.patch
          14 kB
          Rajesh Balamohan
        4. TEZ-1911.1.patch
          15 kB
          Rajesh Balamohan

        Issue Links

          Activity

            People

              rajesh.balamohan Rajesh Balamohan
              rajesh.balamohan Rajesh Balamohan
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: