Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-2972

Avoid task rescheduling when a node turns unhealthy

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.7.1, 0.8.2
    • None
    • None
    • Incompatible change

    Description

      This is similar to MAPREDUCE-6119. Sometimes reacting to a node update event can cause more harm than good. For example, an UNHEALTHY node may be able to shuffle just fine. Therefore obsoleting the output of tasks that ran on that node and re-running them simply adds more overhead to the job with no benefit. It would be nice to be able to configure Tez to ignore node update events if desired.

      Attachments

        1. TEZ-2972.001.patch
          7 kB
          Jason Darrell Lowe
        2. TEZ-2972.002.patch
          10 kB
          Jason Darrell Lowe
        3. TEZ-2972.003.addendum.patch
          0.4 kB
          Jason Darrell Lowe
        4. TEZ-2972.003.patch
          10 kB
          Jason Darrell Lowe
        5. TEZ-2972-branch-0.7.001.patch
          8 kB
          Jason Darrell Lowe

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. TEZ-3051 Vertex failed with invalid event DAG_VERTEX_RERUNNING at SUCCEEDED

          • Major - Major loss of function.
          • Open

          Activity

            People

              jlowe Jason Darrell Lowe
              jlowe Jason Darrell Lowe
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: