Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-2217

The min-held-containers being released prematurely

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.6.0, 0.5.3, 0.7.0
    • Fix Version/s: 0.5.4
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The min-held containers constraint is respected during query idle times, but is not respected when a query is actually in motion.

      The AM releases unused containers during dag execution without checking for min-held containers.

      2015-03-20 15:41:53,475 INFO [DelayedContainerManager] rm.YarnTaskSchedulerService: Container's idle timeout expired. Releasing container, containerId=container_1424502260528_1348_01_000013, containerExpiryTime=1426891313264, idleTimeoutMin=5000
      2015-03-20 15:41:53,475 INFO [DelayedContainerManager] rm.YarnTaskSchedulerService: Releasing unused container: container_1424502260528_1348_01_000013
      

      This is actually useful only after the AM has received a soft pre-emption message, doing it on an idle cluster slows down one of the most common query patterns in BI systems.

      create temporary table smalltable as ...; 
      select ... bigtable JOIN smalltable ON ...;
      

      The smaller query in the beginning throws away the pre-warmed capacity.

        Attachments

        1. TEZ-2217.txt.bz2
          1.65 MB
          Gopal Vijayaraghavan
        2. TEZ-2217.1.patch
          10 kB
          Bikas Saha
        3. TEZ-2217-debug.txt.bz2
          2.51 MB
          Gopal Vijayaraghavan
        4. TEZ-2217.2.patch
          14 kB
          Bikas Saha
        5. TEZ-2217.3.patch
          14 kB
          Bikas Saha
        6. TEZ-2217.4.patch
          14 kB
          Bikas Saha

          Activity

            People

            • Assignee:
              bikassaha Bikas Saha
              Reporter:
              gopalv Gopal Vijayaraghavan
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: