Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-3274

Vertex with MRInput and broadcast input does not respect slow start

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.9.0
    • None
    • None

    Description

      Vertices with shuffle input and MRInput choose RootInputVertexManager (and not ShuffleVertexManager) and start containers and tasks immediately. In this scenario, resources can be wasted since they do not respect tez.shuffle-vertex-manager.min-src-fraction tez.shuffle-vertex-manager.max-src-fraction.

      Attachments

        1. TEZ-3274.009.patch
          49 kB
          Eric Badger
        2. TEZ-3274.008.patch
          50 kB
          Eric Badger
        3. TEZ-3274.007.patch
          50 kB
          Eric Badger
        4. TEZ-3274.006.patch
          51 kB
          Eric Badger
        5. TEZ-3274.005.patch
          52 kB
          Eric Badger
        6. TEZ-3274.004.patch
          50 kB
          Eric Badger
        7. TEZ-3274.003.patch
          8 kB
          Eric Badger
        8. TEZ-3274.002.patch
          79 kB
          Eric Badger
        9. TEZ-3274.001.patch
          79 kB
          Eric Badger

        Issue Links

        breaks

        Bug - A problem which impairs or prevents the functions of the product. TEZ-3792 RootInputVertexManager doesn't drain queued source task completed events

        • Blocker - Blocks development and/or testing work, production could not run
        • Closed
        is related to

        Bug - A problem which impairs or prevents the functions of the product. TEZ-3781 Add TEZ_ROOT_INPUT_VERTEX_MANAGER_MIN_SRC_FRACTION to DeprecatedKeys

        • Major - Major loss of function.
        • Open
        relates to

        Sub-task - The sub-task of the issue TEZ-394 Better scheduling for uneven DAGs

        • Major - Major loss of function.
        • Patch Available

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ebadger Eric Badger
            jeagles Jonathan Turner Eagles
            Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment