Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-2972

Avoid task rescheduling when a node turns unhealthy

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.7.1, 0.8.2
    • None
    • None
    • Incompatible change

    Description

      This is similar to MAPREDUCE-6119. Sometimes reacting to a node update event can cause more harm than good. For example, an UNHEALTHY node may be able to shuffle just fine. Therefore obsoleting the output of tasks that ran on that node and re-running them simply adds more overhead to the job with no benefit. It would be nice to be able to configure Tez to ignore node update events if desired.

      Attachments

        1. TEZ-2972.001.patch
          7 kB
          Jason Darrell Lowe
        2. TEZ-2972.002.patch
          10 kB
          Jason Darrell Lowe
        3. TEZ-2972.003.addendum.patch
          0.4 kB
          Jason Darrell Lowe
        4. TEZ-2972.003.patch
          10 kB
          Jason Darrell Lowe
        5. TEZ-2972-branch-0.7.001.patch
          8 kB
          Jason Darrell Lowe

        Issue Links

        is related to

        Bug - A problem which impairs or prevents the functions of the product. TEZ-3051 Vertex failed with invalid event DAG_VERTEX_RERUNNING at SUCCEEDED

        • Major - Major loss of function.
        • Open

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            jlowe Jason Darrell Lowe
            jlowe Jason Darrell Lowe
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment