Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-3274

Vertex with MRInput and broadcast input does not respect slow start

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.9.0
    • None
    • None

    Description

      Vertices with shuffle input and MRInput choose RootInputVertexManager (and not ShuffleVertexManager) and start containers and tasks immediately. In this scenario, resources can be wasted since they do not respect tez.shuffle-vertex-manager.min-src-fraction tez.shuffle-vertex-manager.max-src-fraction.

      Attachments

        1. TEZ-3274.001.patch
          79 kB
          Eric Badger
        2. TEZ-3274.002.patch
          79 kB
          Eric Badger
        3. TEZ-3274.003.patch
          8 kB
          Eric Badger
        4. TEZ-3274.004.patch
          50 kB
          Eric Badger
        5. TEZ-3274.005.patch
          52 kB
          Eric Badger
        6. TEZ-3274.006.patch
          51 kB
          Eric Badger
        7. TEZ-3274.007.patch
          50 kB
          Eric Badger
        8. TEZ-3274.008.patch
          50 kB
          Eric Badger
        9. TEZ-3274.009.patch
          49 kB
          Eric Badger

        Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. TEZ-3792 RootInputVertexManager doesn't drain queued source task completed events

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed
          is related to

          Bug - A problem which impairs or prevents the functions of the product. TEZ-3781 Add TEZ_ROOT_INPUT_VERTEX_MANAGER_MIN_SRC_FRACTION to DeprecatedKeys

          • Major - Major loss of function.
          • Open
          relates to

          Sub-task - The sub-task of the issue TEZ-394 Better scheduling for uneven DAGs

          • Major - Major loss of function.
          • Patch Available

          Activity

            People

              ebadger Eric Badger
              jeagles Jonathan Turner Eagles
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: