Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-3716

Allow attempt retries to be treated the same as the first attempt

    Details

    • Type: Task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.0
    • Component/s: None
    • Labels:
      None
    • Target Version/s:

      Description

      Essentially - don't change attempt priority due to a failure.

      1. TEZ-3716.01.patch
        11 kB
        Siddharth Seth
      2. TEZ-3716.02.patch
        13 kB
        Siddharth Seth

        Issue Links

          Activity

          Hide
          sseth Siddharth Seth added a comment -

          Switched from changing the dagschedulers to making the change in TaskAttemptImpl itself, which is where the final priority is selected.
          Little larger than expected since this introduces a Config in Vertex to avoid reading the config from hadoop configuration for each attempt.

          Show
          sseth Siddharth Seth added a comment - Switched from changing the dagschedulers to making the change in TaskAttemptImpl itself, which is where the final priority is selected. Little larger than expected since this introduces a Config in Vertex to avoid reading the config from hadoop configuration for each attempt.
          Hide
          sseth Siddharth Seth added a comment -
          Show
          sseth Siddharth Seth added a comment - Rajesh Balamohan , Zhiyuan Yang , Harish Jaiprakash - please review.
          Hide
          tezqa TezQA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12867682/TEZ-3716.01.patch
          against master revision a55fe80.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          -1 javac. The patch appears to cause the build to fail.

          Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/2433//console

          This message is automatically generated.

          Show
          tezqa TezQA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12867682/TEZ-3716.01.patch against master revision a55fe80. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. -1 javac . The patch appears to cause the build to fail. Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/2433//console This message is automatically generated.
          Hide
          rajesh.balamohan Rajesh Balamohan added a comment -

          Patch lgtm. +1 Pending jenkins.

          In TaskAttemptImpl.java, hungIntervalMax, isSpeculationEnabled are places which can be from VertexConfig as well i believe. getCompletedLogsUrl can continue to use conf. Can be addressed in different jira.

          Show
          rajesh.balamohan Rajesh Balamohan added a comment - Patch lgtm. +1 Pending jenkins. In TaskAttemptImpl.java, hungIntervalMax, isSpeculationEnabled are places which can be from VertexConfig as well i believe. getCompletedLogsUrl can continue to use conf. Can be addressed in different jira.
          Hide
          sseth Siddharth Seth added a comment -

          Re-submitted to jenkins now that TEZ-3715 is committed. Will create a jira to move these additional parameters into VertexConf.
          Will commit this if jenkins comes back clean.

          Show
          sseth Siddharth Seth added a comment - Re-submitted to jenkins now that TEZ-3715 is committed. Will create a jira to move these additional parameters into VertexConf. Will commit this if jenkins comes back clean.
          Hide
          tezqa TezQA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12867682/TEZ-3716.01.patch
          against master revision dd9c517.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in :
          org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter
          org.apache.tez.dag.app.dag.impl.TestTaskImpl

          Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/2439//testReport/
          Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/2439//console

          This message is automatically generated.

          Show
          tezqa TezQA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12867682/TEZ-3716.01.patch against master revision dd9c517. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 findbugs . The patch does not introduce any new Findbugs (version 3.0.1) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in : org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter org.apache.tez.dag.app.dag.impl.TestTaskImpl Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/2439//testReport/ Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/2439//console This message is automatically generated.
          Hide
          sseth Siddharth Seth added a comment -

          Updated patch to fix the failing unit tests.

          Show
          sseth Siddharth Seth added a comment - Updated patch to fix the failing unit tests.
          Hide
          tezqa TezQA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12867920/TEZ-3716.02.patch
          against master revision dd9c517.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in :
          org.apache.tez.dag.app.rm.TestTaskSchedulerManager

          Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/2440//testReport/
          Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/2440//console

          This message is automatically generated.

          Show
          tezqa TezQA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12867920/TEZ-3716.02.patch against master revision dd9c517. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 findbugs . The patch does not introduce any new Findbugs (version 3.0.1) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in : org.apache.tez.dag.app.rm.TestTaskSchedulerManager Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/2440//testReport/ Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/2440//console This message is automatically generated.
          Hide
          sseth Siddharth Seth added a comment -

          Test failure is not related. Intermittent bind failure?
          Committing. Thanks for the review Rajesh Balamohan

          Show
          sseth Siddharth Seth added a comment - Test failure is not related. Intermittent bind failure? Committing. Thanks for the review Rajesh Balamohan

            People

            • Assignee:
              sseth Siddharth Seth
              Reporter:
              sseth Siddharth Seth
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development