[MAPREDUCE-6795] Update the document for JobConf#setNumReduceTasks - ASF JIRA

Hide

Permalink

Yiqun Lin added a comment - 30/Oct/16 06:05

Thanks Akira Ajisaka for reporting this. I'd like to make a quick fix. But it seems that I am not a contributor of MAPREDUCE now and I can't upload my patch. The patch file created in my local:

diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobConf.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobConf.java
index f2b0aae..c8b0458 100644
--- a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobConf.java
+++ b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobConf.java
@@ -1347,9 +1347,9 @@ public void setReduceSpeculativeExecution(boolean speculativeExecution) {
    * <b id="NoOfReduces">How many reduces?</b>
    * 
    * <p>The right number of reduces seems to be <code>0.95</code> or 
-   * <code>1.75</code> multiplied by (&lt;<i>no. of nodes</i>&gt; * 
-   * <a href="{@docRoot}/../mapred-default.html#mapreduce.tasktracker.reduce.tasks.maximum">
-   * mapreduce.tasktracker.reduce.tasks.maximum</a>).
+   * <code>1.75</code> multiplied by (
+   * <a href="{@docRoot}/../mapred-default.html#mapreduce.job.running.reduce.limit">
+   * mapreduce.job.running.reduce.limit</a>).
    * </p>
    * 
    * <p>With <code>0.95</code> all of the reduces can launch immediately and

I think the value numNodes * mapreduce.tasktracker.reduce.tasks.maximum in MRV1 is equal to the current config mapreduce.job.running.reduce.limit. Correct me if I am wrong. Thanks!

Show

Yiqun Lin added a comment - 30/Oct/16 06:05 Thanks Akira Ajisaka for reporting this. I'd like to make a quick fix. But it seems that I am not a contributor of MAPREDUCE now and I can't upload my patch. The patch file created in my local: diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobConf.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobConf.java index f2b0aae..c8b0458 100644 --- a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobConf.java +++ b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobConf.java @@ -1347,9 +1347,9 @@ public void setReduceSpeculativeExecution( boolean speculativeExecution) { * How many reduces? * * The right number of reduces seems to be <code>0.95</code> or - * <code>1.75</code> multiplied by (<no. of nodes> * - * <a href= "{@docRoot}/../mapred- default .html#mapreduce.tasktracker.reduce.tasks.maximum" > - * mapreduce.tasktracker.reduce.tasks.maximum</a>). + * <code>1.75</code> multiplied by ( + * <a href= "{@docRoot}/../mapred- default .html#mapreduce.job.running.reduce.limit" > + * mapreduce.job.running.reduce.limit</a>). * * * With <code>0.95</code> all of the reduces can launch immediately and I think the value numNodes * mapreduce.tasktracker.reduce.tasks.maximum in MRV1 is equal to the current config mapreduce.job.running.reduce.limit . Correct me if I am wrong. Thanks!

Hide

Permalink

Akira Ajisaka added a comment - 31/Oct/16 03:14

I'm thinking it's more difficult to tune the value in MRv2 than MRv1.

I think the value numNodes * mapreduce.tasktracker.reduce.tasks.maximum in MRV1 is equal to the current config mapreduce.job.running.reduce.limit.

mapreduce.job.running.reduce.limit is an optional parameter (~~MAPREDUCE-5583~~). The number of reduce tasks is limited to "yarn.nodemanager.resource.memory-mb / mapreduce.reduce.memory.mb", but the resource is shared by map tasks and other applications. Therefore the limit of the number of reduce tasks becomes smaller than "yarn.nodemanager.resource.memory-mb / mapreduce.reduce.memory.mb" if some map tasks or other applications are running.

After all, "multiplied by (<available memory for reduce tasks> / mapreduce.reduce.memory.mb)" is good to me.

Show

Akira Ajisaka added a comment - 31/Oct/16 03:14 I'm thinking it's more difficult to tune the value in MRv2 than MRv1. I think the value numNodes * mapreduce.tasktracker.reduce.tasks.maximum in MRV1 is equal to the current config mapreduce.job.running.reduce.limit. mapreduce.job.running.reduce.limit is an optional parameter ( MAPREDUCE-5583 ). The number of reduce tasks is limited to "yarn.nodemanager.resource.memory-mb / mapreduce.reduce.memory.mb", but the resource is shared by map tasks and other applications. Therefore the limit of the number of reduce tasks becomes smaller than "yarn.nodemanager.resource.memory-mb / mapreduce.reduce.memory.mb" if some map tasks or other applications are running. After all, "multiplied by (<available memory for reduce tasks> / mapreduce.reduce.memory.mb)" is good to me.

Hide

Permalink

Yiqun Lin added a comment - 31/Oct/16 05:26

Thanks Akira Ajisaka for sharing your thought.

After all, "multiplied by (<available memory for reduce tasks> / mapreduce.reduce.memory.mb)" is good to me.

LGTM too. Attach a patch for this.

Show

Yiqun Lin added a comment - 31/Oct/16 05:26 Thanks Akira Ajisaka for sharing your thought. After all, "multiplied by (<available memory for reduce tasks> / mapreduce.reduce.memory.mb)" is good to me. LGTM too. Attach a patch for this.

Hide

Permalink

Hadoop QA added a comment - 31/Oct/16 05:44

-1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	0m 16s	Docker mode activated.
+1	@author	0m 0s	The patch does not contain any @author tags.
-1	test4tests	0m 0s	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
+1	mvninstall	6m 48s	trunk passed
+1	compile	0m 23s	trunk passed
+1	checkstyle	0m 17s	trunk passed
+1	mvnsite	0m 29s	trunk passed
+1	mvneclipse	0m 13s	trunk passed
+1	findbugs	0m 48s	trunk passed
+1	javadoc	0m 21s	trunk passed
+1	mvninstall	0m 22s	the patch passed
+1	compile	0m 20s	the patch passed
+1	javac	0m 20s	the patch passed
+1	checkstyle	0m 15s	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core: The patch generated 0 new + 90 unchanged - 1 fixed = 90 total (was 91)
+1	mvnsite	0m 25s	the patch passed
+1	mvneclipse	0m 10s	the patch passed
+1	whitespace	0m 0s	The patch has no whitespace issues.
+1	findbugs	0m 52s	the patch passed
+1	javadoc	0m 18s	the patch passed
+1	unit	2m 42s	hadoop-mapreduce-client-core in the patch passed.
+1	asflicense	0m 17s	The patch does not generate ASF License warnings.
		15m 51s

Subsystem	Report/Notes
Docker	Image:yetus/hadoop:9560f25
JIRA Patch URL	https://issues.apache.org/jira/secure/attachment/12836117/MAPREDUCE-6795.001.patch
JIRA Issue	~~MAPREDUCE-6795~~
Optional Tests	asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
uname	Linux 45a40af43d2d 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	/testptch/hadoop/patchprocess/precommit/personality/provided.sh
git revision	trunk / e0bebbb
Default Java	1.8.0_101
findbugs	v3.0.0
Test Results	https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6786/testReport/
modules	C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
Console output	https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6786/console
Powered by	Apache Yetus 0.3.0 http://yetus.apache.org

This message was automatically generated.

Show

Hadoop QA added a comment - 31/Oct/16 05:44 -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 48s trunk passed +1 compile 0m 23s trunk passed +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 29s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 48s trunk passed +1 javadoc 0m 21s trunk passed +1 mvninstall 0m 22s the patch passed +1 compile 0m 20s the patch passed +1 javac 0m 20s the patch passed +1 checkstyle 0m 15s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core: The patch generated 0 new + 90 unchanged - 1 fixed = 90 total (was 91) +1 mvnsite 0m 25s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 52s the patch passed +1 javadoc 0m 18s the patch passed +1 unit 2m 42s hadoop-mapreduce-client-core in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 15m 51s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12836117/MAPREDUCE-6795.001.patch JIRA Issue MAPREDUCE-6795 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 45a40af43d2d 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e0bebbb Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6786/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6786/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.

Hide

Permalink

Yiqun Lin added a comment - 31/Oct/16 08:28

Reupload the v001 patch. I think it will be better to explain the available memory for reduce tasks here. Make a minor change based on the original patch.

Show

Yiqun Lin added a comment - 31/Oct/16 08:28 Reupload the v001 patch. I think it will be better to explain the available memory for reduce tasks here. Make a minor change based on the original patch.

Hide

Permalink

Akira Ajisaka added a comment - 31/Oct/16 08:48

Thanks Yiqun Lin for updating the patch.

<a href="{@docRoot}/../mapred-default.html#mapreduce.reduce.memory.mb">

Would you fix this dead link as follows?

<a href="{@docRoot}/../hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml#mapreduce.reduce.memory.mb">

Show

Akira Ajisaka added a comment - 31/Oct/16 08:48 Thanks Yiqun Lin for updating the patch. <a href= "{@docRoot}/../mapred- default .html#mapreduce.reduce.memory.mb" > Would you fix this dead link as follows? <a href= "{@docRoot}/../hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred- default .xml#mapreduce.reduce.memory.mb" >

Hide

Permalink

Hadoop QA added a comment - 31/Oct/16 08:54

-1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	0m 23s	Docker mode activated.
+1	@author	0m 0s	The patch does not contain any @author tags.
-1	test4tests	0m 0s	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
+1	mvninstall	6m 42s	trunk passed
+1	compile	0m 23s	trunk passed
+1	checkstyle	0m 17s	trunk passed
+1	mvnsite	0m 28s	trunk passed
+1	mvneclipse	0m 13s	trunk passed
+1	findbugs	0m 47s	trunk passed
+1	javadoc	0m 21s	trunk passed
+1	mvninstall	0m 22s	the patch passed
+1	compile	0m 20s	the patch passed
+1	javac	0m 20s	the patch passed
+1	checkstyle	0m 14s	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core: The patch generated 0 new + 90 unchanged - 1 fixed = 90 total (was 91)
+1	mvnsite	0m 25s	the patch passed
+1	mvneclipse	0m 11s	the patch passed
+1	whitespace	0m 0s	The patch has no whitespace issues.
+1	findbugs	0m 51s	the patch passed
+1	javadoc	0m 19s	the patch passed
+1	unit	2m 41s	hadoop-mapreduce-client-core in the patch passed.
+1	asflicense	0m 15s	The patch does not generate ASF License warnings.
		15m 48s

Subsystem	Report/Notes
Docker	Image:yetus/hadoop:9560f25
JIRA Patch URL	https://issues.apache.org/jira/secure/attachment/12836127/MAPREDUCE-6795.001.patch
JIRA Issue	~~MAPREDUCE-6795~~
Optional Tests	asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
uname	Linux 67014b80dade 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	/testptch/hadoop/patchprocess/precommit/personality/provided.sh
git revision	trunk / f646fe3
Default Java	1.8.0_101
findbugs	v3.0.0
Test Results	https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6788/testReport/
modules	C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
Console output	https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6788/console
Powered by	Apache Yetus 0.3.0 http://yetus.apache.org

This message was automatically generated.

Show

Hadoop QA added a comment - 31/Oct/16 08:54 -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 23s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 42s trunk passed +1 compile 0m 23s trunk passed +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 28s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 47s trunk passed +1 javadoc 0m 21s trunk passed +1 mvninstall 0m 22s the patch passed +1 compile 0m 20s the patch passed +1 javac 0m 20s the patch passed +1 checkstyle 0m 14s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core: The patch generated 0 new + 90 unchanged - 1 fixed = 90 total (was 91) +1 mvnsite 0m 25s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 51s the patch passed +1 javadoc 0m 19s the patch passed +1 unit 2m 41s hadoop-mapreduce-client-core in the patch passed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 15m 48s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12836127/MAPREDUCE-6795.001.patch JIRA Issue MAPREDUCE-6795 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 67014b80dade 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f646fe3 Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6788/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6788/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.

Hide

Permalink

Yiqun Lin added a comment - 31/Oct/16 09:33

Thanks Akira Ajisaka for the quick review and comments! Post the new patch to address the comment.

Show

Yiqun Lin added a comment - 31/Oct/16 09:33 Thanks Akira Ajisaka for the quick review and comments! Post the new patch to address the comment.

Hide

Permalink

Hadoop QA added a comment - 31/Oct/16 10:00

-1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	0m 13s	Docker mode activated.
+1	@author	0m 0s	The patch does not contain any @author tags.
-1	test4tests	0m 0s	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
+1	mvninstall	7m 1s	trunk passed
+1	compile	0m 24s	trunk passed
+1	checkstyle	0m 17s	trunk passed
+1	mvnsite	0m 30s	trunk passed
+1	mvneclipse	0m 12s	trunk passed
+1	findbugs	0m 49s	trunk passed
+1	javadoc	0m 22s	trunk passed
+1	mvninstall	0m 25s	the patch passed
+1	compile	0m 20s	the patch passed
+1	javac	0m 20s	the patch passed
+1	checkstyle	0m 14s	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core: The patch generated 0 new + 90 unchanged - 1 fixed = 90 total (was 91)
+1	mvnsite	0m 26s	the patch passed
+1	mvneclipse	0m 11s	the patch passed
+1	whitespace	0m 0s	The patch has no whitespace issues.
+1	findbugs	0m 55s	the patch passed
-1	javadoc	0m 18s	hadoop-mapreduce-client-core in the patch failed.
+1	unit	2m 46s	hadoop-mapreduce-client-core in the patch passed.
+1	asflicense	0m 15s	The patch does not generate ASF License warnings.
		16m 17s

Subsystem	Report/Notes
Docker	Image:yetus/hadoop:9560f25
JIRA Patch URL	https://issues.apache.org/jira/secure/attachment/12836131/MAPREDUCE-6795.002.patch
JIRA Issue	~~MAPREDUCE-6795~~
Optional Tests	asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
uname	Linux 27517cf4470e 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	/testptch/hadoop/patchprocess/precommit/personality/provided.sh
git revision	trunk / f646fe3
Default Java	1.8.0_101
findbugs	v3.0.0
javadoc	https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6789/artifact/patchprocess/patch-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt
Test Results	https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6789/testReport/
modules	C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
Console output	https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6789/console
Powered by	Apache Yetus 0.3.0 http://yetus.apache.org

This message was automatically generated.

Show

Hadoop QA added a comment - 31/Oct/16 10:00 -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 1s trunk passed +1 compile 0m 24s trunk passed +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 30s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 0m 49s trunk passed +1 javadoc 0m 22s trunk passed +1 mvninstall 0m 25s the patch passed +1 compile 0m 20s the patch passed +1 javac 0m 20s the patch passed +1 checkstyle 0m 14s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core: The patch generated 0 new + 90 unchanged - 1 fixed = 90 total (was 91) +1 mvnsite 0m 26s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 55s the patch passed -1 javadoc 0m 18s hadoop-mapreduce-client-core in the patch failed. +1 unit 2m 46s hadoop-mapreduce-client-core in the patch passed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 16m 17s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12836131/MAPREDUCE-6795.002.patch JIRA Issue MAPREDUCE-6795 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 27517cf4470e 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f646fe3 Default Java 1.8.0_101 findbugs v3.0.0 javadoc https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6789/artifact/patchprocess/patch-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6789/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6789/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.

Hide

Permalink

Akira Ajisaka added a comment - 31/Oct/16 10:36

   * <a href="{@docRoot}/../hadoop-mapreduce-client/hadoop-mapreduce-client-
   * core/mapred-default.xml#mapreduce.reduce.memory.mb">

Would you make this one line to remove unnecessary whitespace from the link? I'm +1 if that is addressed.

Show

Akira Ajisaka added a comment - 31/Oct/16 10:36 * <a href="{@docRoot}/../hadoop-mapreduce-client/hadoop-mapreduce-client- * core/mapred- default .xml#mapreduce.reduce.memory.mb"> Would you make this one line to remove unnecessary whitespace from the link? I'm +1 if that is addressed.

Hide

Permalink

Yiqun Lin added a comment - 31/Oct/16 11:16

I am sorry for ignoring that, reupload v002 patch again.

Show

Yiqun Lin added a comment - 31/Oct/16 11:16 I am sorry for ignoring that, reupload v002 patch again.

Hide

Permalink

Hadoop QA added a comment - 31/Oct/16 11:39

-1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	0m 17s	Docker mode activated.
+1	@author	0m 0s	The patch does not contain any @author tags.
-1	test4tests	0m 0s	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
+1	mvninstall	8m 54s	trunk passed
+1	compile	0m 32s	trunk passed
+1	checkstyle	0m 21s	trunk passed
+1	mvnsite	0m 38s	trunk passed
+1	mvneclipse	0m 15s	trunk passed
+1	findbugs	1m 7s	trunk passed
+1	javadoc	0m 25s	trunk passed
+1	mvninstall	0m 32s	the patch passed
+1	compile	0m 30s	the patch passed
+1	javac	0m 30s	the patch passed
-1	checkstyle	0m 20s	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core: The patch generated 1 new + 90 unchanged - 1 fixed = 91 total (was 91)
+1	mvnsite	0m 33s	the patch passed
+1	mvneclipse	0m 11s	the patch passed
+1	whitespace	0m 0s	The patch has no whitespace issues.
+1	findbugs	0m 54s	the patch passed
+1	javadoc	0m 18s	the patch passed
+1	unit	2m 46s	hadoop-mapreduce-client-core in the patch passed.
+1	asflicense	0m 17s	The patch does not generate ASF License warnings.
		19m 32s

Subsystem	Report/Notes
Docker	Image:yetus/hadoop:9560f25
JIRA Patch URL	https://issues.apache.org/jira/secure/attachment/12836137/MAPREDUCE-6795.002.patch
JIRA Issue	~~MAPREDUCE-6795~~
Optional Tests	asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
uname	Linux 1625eccd3400 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	/testptch/hadoop/patchprocess/precommit/personality/provided.sh
git revision	trunk / f646fe3
Default Java	1.8.0_101
findbugs	v3.0.0
checkstyle	https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6790/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt
Test Results	https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6790/testReport/
modules	C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
Console output	https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6790/console
Powered by	Apache Yetus 0.3.0 http://yetus.apache.org

This message was automatically generated.

Show

Hadoop QA added a comment - 31/Oct/16 11:39 -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 8m 54s trunk passed +1 compile 0m 32s trunk passed +1 checkstyle 0m 21s trunk passed +1 mvnsite 0m 38s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 7s trunk passed +1 javadoc 0m 25s trunk passed +1 mvninstall 0m 32s the patch passed +1 compile 0m 30s the patch passed +1 javac 0m 30s the patch passed -1 checkstyle 0m 20s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core: The patch generated 1 new + 90 unchanged - 1 fixed = 91 total (was 91) +1 mvnsite 0m 33s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 54s the patch passed +1 javadoc 0m 18s the patch passed +1 unit 2m 46s hadoop-mapreduce-client-core in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 19m 32s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12836137/MAPREDUCE-6795.002.patch JIRA Issue MAPREDUCE-6795 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 1625eccd3400 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f646fe3 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6790/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6790/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6790/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.

Hide

Permalink

Akira Ajisaka added a comment - 01/Nov/16 11:50

+1, committing this.

Show

Akira Ajisaka added a comment - 01/Nov/16 11:50 +1, committing this.

Hide

Permalink

Akira Ajisaka added a comment - 01/Nov/16 11:53

Committed this to trunk, branch-2, and branch-2.8. Thanks Yiqun Lin for the contribution!

Show

Akira Ajisaka added a comment - 01/Nov/16 11:53 Committed this to trunk, branch-2, and branch-2.8. Thanks Yiqun Lin for the contribution!

Hide

Permalink

Hudson added a comment - 01/Nov/16 12:20

SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10743 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10743/)
~~MAPREDUCE-6795~~. Update the document for JobConf#setNumReduceTasks. (aajisaka: rev 235f39b7c44c5907bd4de063892905577c35cd80)

(edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobConf.java

Show

Hudson added a comment - 01/Nov/16 12:20 SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10743 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10743/ ) MAPREDUCE-6795 . Update the document for JobConf#setNumReduceTasks. (aajisaka: rev 235f39b7c44c5907bd4de063892905577c35cd80) (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobConf.java

Hide

Permalink

Yiqun Lin added a comment - 02/Nov/16 12:44

Thanks Akira Ajisaka for the patient review and commit!

Show

Yiqun Lin added a comment - 02/Nov/16 12:44 Thanks Akira Ajisaka for the patient review and commit!

Update the document for JobConf#setNumReduceTasks

Details

Description

Attachments

Issue Links

Activity

People

Dates

Development

Agile