Details
-
Type:
Bug
-
Status: Resolved
-
Priority:
Minor
-
Resolution: Fixed
-
Affects Version/s: 2.8.0
-
Fix Version/s: 2.8.0, 3.0.0-alpha2
-
Component/s: mr-am
-
Labels:None
-
Target Version/s:
-
Hadoop Flags:Reviewed
Description
MAPREDUCE-6242 makes task status update interval configurable to ease the pressure on MR AM to process status updates, but it did not ensure that mapreduce.task.timeout is no smaller than the configured value of task report interval.
Issue Links
- is part of
-
MAPREDUCE-6242
Progress report log is incredibly excessive in application master
-
- Resolved
-
Activity
- All
- Comments
- Work Log
- History
- Activity
- Transitions
 -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 20s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| +1 | mvninstall | 10m 44s | trunk passed |
| +1 | compile | 0m 35s | trunk passed |
| +1 | checkstyle | 0m 24s | trunk passed |
| +1 | mvnsite | 0m 44s | trunk passed |
| +1 | mvneclipse | 0m 21s | trunk passed |
| +1 | findbugs | 0m 52s | trunk passed |
| +1 | javadoc | 0m 21s | trunk passed |
| +1 | mvninstall | 0m 32s | the patch passed |
| +1 | compile | 0m 31s | the patch passed |
| +1 | javac | 0m 31s | the patch passed |
| -1 | checkstyle | 0m 18s | hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app: The patch generated 1 new + 3 unchanged - 1 fixed = 4 total (was 4) |
| +1 | mvnsite | 0m 38s | the patch passed |
| +1 | mvneclipse | 0m 18s | the patch passed |
| +1 | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 | findbugs | 0m 59s | the patch passed |
| +1 | javadoc | 0m 19s | the patch passed |
| +1 | unit | 10m 35s | hadoop-mapreduce-client-app in the patch passed. |
| +1 | asflicense | 0m 23s | The patch does not generate ASF License warnings. |
| 29m 47s |
| Subsystem | Report/Notes |
|---|---|
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12819237/mapreduce6740.001.patch |
| JIRA Issue | |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle |
| uname | Linux bad9082b10d9 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / e340064 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| checkstyle | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6633/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt |
| Test Results | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6633/testReport/ |
| modules | C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app |
| Console output | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6633/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
Uploading new patch to fix the checkstyle issue reported.
 +1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 27s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| +1 | mvninstall | 7m 17s | trunk passed |
| +1 | compile | 0m 23s | trunk passed |
| +1 | checkstyle | 0m 15s | trunk passed |
| +1 | mvnsite | 0m 28s | trunk passed |
| +1 | mvneclipse | 0m 15s | trunk passed |
| +1 | findbugs | 0m 36s | trunk passed |
| +1 | javadoc | 0m 15s | trunk passed |
| +1 | mvninstall | 0m 21s | the patch passed |
| +1 | compile | 0m 20s | the patch passed |
| +1 | javac | 0m 20s | the patch passed |
| +1 | checkstyle | 0m 13s | hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app: The patch generated 0 new + 3 unchanged - 1 fixed = 3 total (was 4) |
| +1 | mvnsite | 0m 25s | the patch passed |
| +1 | mvneclipse | 0m 13s | the patch passed |
| +1 | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 | findbugs | 0m 40s | the patch passed |
| +1 | javadoc | 0m 12s | the patch passed |
| +1 | unit | 8m 49s | hadoop-mapreduce-client-app in the patch passed. |
| +1 | asflicense | 0m 15s | The patch does not generate ASF License warnings. |
| 21m 59s |
| Subsystem | Report/Notes |
|---|---|
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12819429/mapreduce6740.002.patch |
| JIRA Issue | |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle |
| uname | Linux 2638bcf1426f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / ecff7d0 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| Test Results | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6637/testReport/ |
| modules | C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app |
| Console output | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6637/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
The value of taskTimeout should be larger than the progress-report-interval. I wonder if it is okay to enforce at least twice the progress-interval.
Uploaded a new patch to address Karthik's comment. This patch
1) enforces task timeout to be as least twice as long as progress interval
2) sets the default value of progress report interval to 1% of the default value of task timeout, which is still 3000 milliseconds.
Regardless of what value the user set the progress-report-interval to, the task timeout will always be Max(2*progress-report-interval, configured value of task timeout).
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 17s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| 0 | mvndep | 0m 53s | Maven dependency ordering for branch |
| +1 | mvninstall | 7m 37s | trunk passed |
| +1 | compile | 1m 42s | trunk passed |
| +1 | checkstyle | 0m 35s | trunk passed |
| +1 | mvnsite | 0m 59s | trunk passed |
| +1 | mvneclipse | 0m 28s | trunk passed |
| +1 | findbugs | 1m 29s | trunk passed |
| +1 | javadoc | 0m 36s | trunk passed |
| 0 | mvndep | 0m 8s | Maven dependency ordering for patch |
| +1 | mvninstall | 0m 44s | the patch passed |
| +1 | compile | 1m 35s | the patch passed |
| +1 | javac | 1m 35s | the patch passed |
| -1 | checkstyle | 0m 33s | hadoop-mapreduce-project/hadoop-mapreduce-client: The patch generated 2 new + 723 unchanged - 3 fixed = 725 total (was 726) |
| +1 | mvnsite | 0m 55s | the patch passed |
| +1 | mvneclipse | 0m 24s | the patch passed |
| +1 | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 | findbugs | 1m 36s | the patch passed |
| +1 | javadoc | 0m 34s | the patch passed |
| +1 | unit | 2m 6s | hadoop-mapreduce-client-core in the patch passed. |
| -1 | unit | 9m 13s | hadoop-mapreduce-client-app in the patch failed. |
| +1 | asflicense | 0m 17s | The patch does not generate ASF License warnings. |
| 33m 29s |
| Reason | Tests |
|---|---|
| Failed junit tests | hadoop.mapreduce.v2.app.TestFail |
| Timed out junit tests | org.apache.hadoop.mapreduce.v2.app.launcher.TestContainerLauncher |
| Subsystem | Report/Notes |
|---|---|
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12823386/mapreduce6740.003.patch |
| JIRA Issue | |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle |
| uname | Linux 8e8e48fae62a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 23c6e3c |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| checkstyle | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6670/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt |
| unit | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6670/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt |
| unit test logs | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6670/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt |
| Test Results | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6670/testReport/ |
| modules | C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: hadoop-mapreduce-project/hadoop-mapreduce-client |
| Console output | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6670/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
Updating a new patch to fix the checkstyle and unit test issue.
| +1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 11s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. |
| 0 | mvndep | 2m 9s | Maven dependency ordering for branch |
| +1 | mvninstall | 8m 30s | trunk passed |
| +1 | compile | 1m 53s | trunk passed |
| +1 | checkstyle | 0m 39s | trunk passed |
| +1 | mvnsite | 1m 7s | trunk passed |
| +1 | mvneclipse | 0m 31s | trunk passed |
| +1 | findbugs | 1m 41s | trunk passed |
| +1 | javadoc | 0m 42s | trunk passed |
| 0 | mvndep | 0m 8s | Maven dependency ordering for patch |
| +1 | mvninstall | 0m 47s | the patch passed |
| +1 | compile | 1m 49s | the patch passed |
| +1 | javac | 1m 49s | the patch passed |
| +1 | checkstyle | 0m 37s | hadoop-mapreduce-project/hadoop-mapreduce-client: The patch generated 0 new + 733 unchanged - 3 fixed = 733 total (was 736) |
| +1 | mvnsite | 1m 1s | the patch passed |
| +1 | mvneclipse | 0m 26s | the patch passed |
| +1 | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 | findbugs | 1m 52s | the patch passed |
| +1 | javadoc | 0m 35s | the patch passed |
| +1 | unit | 2m 12s | hadoop-mapreduce-client-core in the patch passed. |
| +1 | unit | 9m 14s | hadoop-mapreduce-client-app in the patch passed. |
| +1 | asflicense | 0m 21s | The patch does not generate ASF License warnings. |
| 37m 13s |
| Subsystem | Report/Notes |
|---|---|
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12823731/mapreduce6740.004.patch |
| JIRA Issue | |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle |
| uname | Linux 151b5fea8c6e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 2424911 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| Test Results | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6671/testReport/ |
| modules | C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: hadoop-mapreduce-project/hadoop-mapreduce-client |
| Console output | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6671/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
Per offline discussion with Karthik, the default value of task progress report interval can be set to a portion of the value of task timeout. This gets rid of the default value of task progress interval in MRJobConfig.java. In this patch, the ratio is set to 0.01 so that in cases where both task report interval and task timeout are not set explicitly, the value of task progress report defaults to 3000 milliseconds, preserving its current default value.
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 13m 36s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| 0 | mvndep | 1m 4s | Maven dependency ordering for branch |
| +1 | mvninstall | 9m 11s | trunk passed |
| +1 | compile | 1m 45s | trunk passed |
| +1 | checkstyle | 0m 37s | trunk passed |
| +1 | mvnsite | 1m 1s | trunk passed |
| +1 | mvneclipse | 0m 30s | trunk passed |
| +1 | findbugs | 1m 28s | trunk passed |
| +1 | javadoc | 0m 38s | trunk passed |
| 0 | mvndep | 0m 8s | Maven dependency ordering for patch |
| +1 | mvninstall | 0m 45s | the patch passed |
| +1 | compile | 1m 35s | the patch passed |
| +1 | javac | 1m 35s | the patch passed |
| +1 | checkstyle | 0m 33s | hadoop-mapreduce-project/hadoop-mapreduce-client: The patch generated 0 new + 723 unchanged - 3 fixed = 723 total (was 726) |
| +1 | mvnsite | 0m 56s | the patch passed |
| +1 | mvneclipse | 0m 23s | the patch passed |
| +1 | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 | findbugs | 1m 36s | the patch passed |
| -1 | javadoc | 0m 18s | hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core generated 2 new + 2508 unchanged - 0 fixed = 2510 total (was 2508) |
| -1 | unit | 2m 13s | hadoop-mapreduce-client-core in the patch failed. |
| +1 | unit | 8m 47s | hadoop-mapreduce-client-app in the patch passed. |
| +1 | asflicense | 0m 18s | The patch does not generate ASF License warnings. |
| 48m 23s |
| Reason | Tests |
|---|---|
| Failed junit tests | hadoop.mapred.TestTaskProgressReporter |
This message was automatically generated.
Attaching a new patch to address the unit test failure and java doc warnings.
| +1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 22s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| 0 | mvndep | 0m 8s | Maven dependency ordering for branch |
| +1 | mvninstall | 6m 44s | trunk passed |
| +1 | compile | 1m 34s | trunk passed |
| +1 | checkstyle | 0m 34s | trunk passed |
| +1 | mvnsite | 0m 57s | trunk passed |
| +1 | mvneclipse | 0m 28s | trunk passed |
| +1 | findbugs | 1m 20s | trunk passed |
| +1 | javadoc | 0m 36s | trunk passed |
| 0 | mvndep | 0m 8s | Maven dependency ordering for patch |
| +1 | mvninstall | 0m 43s | the patch passed |
| +1 | compile | 1m 31s | the patch passed |
| +1 | javac | 1m 31s | the patch passed |
| +1 | checkstyle | 0m 32s | hadoop-mapreduce-project/hadoop-mapreduce-client: The patch generated 0 new + 723 unchanged - 3 fixed = 723 total (was 726) |
| +1 | mvnsite | 0m 52s | the patch passed |
| +1 | mvneclipse | 0m 23s | the patch passed |
| +1 | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 | findbugs | 1m 33s | the patch passed |
| +1 | javadoc | 0m 32s | the patch passed |
| +1 | unit | 2m 7s | hadoop-mapreduce-client-core in the patch passed. |
| +1 | unit | 8m 44s | hadoop-mapreduce-client-app in the patch passed. |
| +1 | asflicense | 0m 16s | The patch does not generate ASF License warnings. |
| 30m 48s |
| Subsystem | Report/Notes |
|---|---|
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12824239/mapreduce6740.006.patch |
| JIRA Issue | |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle |
| uname | Linux f6c324e21573 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 8693936 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| Test Results | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6675/testReport/ |
| modules | C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: hadoop-mapreduce-project/hadoop-mapreduce-client |
| Console output | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6675/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
Latest patch looks good. Couple of minor comments:
- TaskHeartbeatHandler: When declaring taskTimeout, avoid setting the value to 5 * 60 * 1000.
- Do we need a test case for when we don't set TASK_REPORT_INTERVAL?
Sorry for the long delay updating this patch. Uploaded a new version to address Karthik's comments.
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 13s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| 0 | mvndep | 0m 8s | Maven dependency ordering for branch |
| +1 | mvninstall | 6m 45s | trunk passed |
| +1 | compile | 1m 38s | trunk passed |
| +1 | checkstyle | 0m 34s | trunk passed |
| +1 | mvnsite | 0m 56s | trunk passed |
| +1 | mvneclipse | 0m 28s | trunk passed |
| +1 | findbugs | 1m 23s | trunk passed |
| +1 | javadoc | 0m 36s | trunk passed |
| 0 | mvndep | 0m 7s | Maven dependency ordering for patch |
| +1 | mvninstall | 0m 44s | the patch passed |
| +1 | compile | 1m 34s | the patch passed |
| +1 | javac | 1m 34s | the patch passed |
| -1 | checkstyle | 0m 34s | hadoop-mapreduce-project/hadoop-mapreduce-client: The patch generated 3 new + 723 unchanged - 3 fixed = 726 total (was 726) |
| +1 | mvnsite | 0m 54s | the patch passed |
| +1 | mvneclipse | 0m 23s | the patch passed |
| +1 | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 | findbugs | 1m 35s | the patch passed |
| +1 | javadoc | 0m 32s | the patch passed |
| +1 | unit | 2m 12s | hadoop-mapreduce-client-core in the patch passed. |
| +1 | unit | 8m 45s | hadoop-mapreduce-client-app in the patch passed. |
| +1 | asflicense | 0m 16s | The patch does not generate ASF License warnings. |
| 31m 2s |
| Subsystem | Report/Notes |
|---|---|
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12828913/mapreduce6740.007.patch |
| JIRA Issue | |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle |
| uname | Linux b6354dedbf01 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / f6f3a44 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| checkstyle | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6724/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt |
| Test Results | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6724/testReport/ |
| modules | C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: hadoop-mapreduce-project/hadoop-mapreduce-client |
| Console output | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6724/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
New patch to address checkstyle issues.
+1 on the latest patch.
Not sure why Jenkins hasn't kicked in. Verified that v7 and v8 versions of the patch differ only in line lengths and that too only in comments.
Committing this based on the Jenkins output of v7...
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10472 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10472/)
MAPREDUCE-6740. Enforce mapreduce.task.timeout to be at least (kasha: rev 537095d13cd38212ed162e0a360bdd9a8bd83498)
- (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
- (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/MRJobConfUtil.java
- (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/TaskHeartbeatHandler.java
- (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/Task.java
- (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestTaskHeartbeatHandler.java
Committed this to trunk and branch-2.
Haibo Chen - the patch does not apply to branch-2.8. Given MR-6242 got committed to 2.8, it would be nice to get this into 2.8 as well. Mind posting a patch for the same?
Yep, will upload a 2.8 patch later today.
Karthik Kambatla Turns out MR-6741 is the dependency and it is not committed to 2.8. What's right action here?
If MAPREDUCE-6741 applies to 2.8 cleanly, we should likely do that. If it does not, could you look into posting the patch for 2.8 there please?
Karthik Kambatla I have submitted a branch-2.8 patch in MR-6741. With that patch in branch-2.8, the latest patch in the jira applies cleanly.
Karthik Kambatla, Jason has committed the branch-2.8 patch in MR-6741. You should be able to commit this patch cleanly to 2.8 now.
Uploading a patch for branch-2.8 only
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 17s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| 0 | mvndep | 0m 13s | Maven dependency ordering for branch |
| +1 | mvninstall | 6m 37s | branch-2.8 passed |
| +1 | compile | 1m 2s | branch-2.8 passed with JDK v1.8.0_101 |
| +1 | compile | 1m 14s | branch-2.8 passed with JDK v1.7.0_111 |
| +1 | checkstyle | 0m 33s | branch-2.8 passed |
| +1 | mvnsite | 0m 59s | branch-2.8 passed |
| +1 | mvneclipse | 0m 29s | branch-2.8 passed |
| +1 | findbugs | 1m 46s | branch-2.8 passed |
| +1 | javadoc | 0m 36s | branch-2.8 passed with JDK v1.8.0_101 |
| +1 | javadoc | 0m 41s | branch-2.8 passed with JDK v1.7.0_111 |
| 0 | mvndep | 0m 9s | Maven dependency ordering for patch |
| +1 | mvninstall | 0m 46s | the patch passed |
| +1 | compile | 0m 57s | the patch passed with JDK v1.8.0_101 |
| +1 | javac | 0m 57s | the patch passed |
| +1 | compile | 1m 13s | the patch passed with JDK v1.7.0_111 |
| +1 | javac | 1m 13s | the patch passed |
| +1 | checkstyle | 0m 30s | hadoop-mapreduce-project/hadoop-mapreduce-client: The patch generated 0 new + 724 unchanged - 3 fixed = 724 total (was 727) |
| +1 | mvnsite | 0m 54s | the patch passed |
| +1 | mvneclipse | 0m 26s | the patch passed |
| +1 | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 | findbugs | 2m 6s | the patch passed |
| +1 | javadoc | 0m 31s | the patch passed with JDK v1.8.0_101 |
| +1 | javadoc | 0m 36s | the patch passed with JDK v1.7.0_111 |
| +1 | unit | 1m 57s | hadoop-mapreduce-client-core in the patch passed with JDK v1.8.0_101. |
| -1 | unit | 9m 6s | hadoop-mapreduce-client-app in the patch failed with JDK v1.8.0_101. |
| +1 | unit | 2m 19s | hadoop-mapreduce-client-core in the patch passed with JDK v1.7.0_111. |
| +1 | unit | 9m 45s | hadoop-mapreduce-client-app in the patch passed with JDK v1.7.0_111. |
| +1 | asflicense | 0m 18s | The patch does not generate ASF License warnings. |
| 47m 16s |
| Reason | Tests |
|---|---|
| JDK v1.8.0_101 Failed junit tests | hadoop.mapreduce.v2.app.TestRecovery |
| Subsystem | Report/Notes |
|---|---|
| Docker | Image:yetus/hadoop:5af2af1 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12831790/mapreduce6740.branch-2.8.patch |
| JIRA Issue | |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle |
| uname | Linux 73c90fa2ebb5 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | branch-2.8 / f1b74a3 |
| Default Java | 1.7.0_111 |
| Multi-JDK versions | /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 |
| findbugs | v3.0.0 |
| unit | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6751/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_101.txt |
| unit test logs | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6751/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_101.txt |
| JDK v1.7.0_111 Test Results | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6751/testReport/ |
| modules | C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: hadoop-mapreduce-project/hadoop-mapreduce-client |
| Console output | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6751/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
The unit test failures are unrelated to this patch.
Just committed to branch-2.8 as well. Thanks for posting the updated patch, Haibo Chen.
MAPREDUCE-6740This message was automatically generated.