Details
-
Sub-task
-
Status: Resolved
-
Minor
-
Resolution: Fixed
-
None
-
None
-
Reviewed
Description
A test with 3 Namenodes and 4 Datanodes in startup, and then adding 2 new Datanodes.
Attachments
Attachments
- HDFS-1686.02.patch
- 10 kB
- Bharat Viswanadham
- HDFS-1686.01.patch
- 10 kB
- Bharat Viswanadham
- HDFS-1686.00.patch
- 12 kB
- Bharat Viswanadham
- h1686_20110303.patch
- 6 kB
- Tsz-wo Sze
- 4358946.patch
- 31 kB
- Tsz-wo Sze
Issue Links
- relates to
-
HDFS-14023 TestBalancerWithMultipleNameNodes#test1OutOf2BlockpoolsWithBlockPoolPolicy times out sometimes
- Open
Activity
Hi szetszwo
I have assigned this Jira to myself, as after uploading the patch 2nd time(to correct few errors), not able to change the status of Jira to PATCH Available.
I have taken your patch, and updated to add few more test cases.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 12s | Docker mode activated. |
Prechecks | |||
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
trunk Compile Tests | |||
+1 | mvninstall | 16m 52s | trunk passed |
+1 | compile | 0m 53s | trunk passed |
+1 | checkstyle | 0m 39s | trunk passed |
+1 | mvnsite | 1m 0s | trunk passed |
+1 | shadedclient | 11m 32s | branch has no errors when building and testing our client artifacts. |
+1 | findbugs | 1m 55s | trunk passed |
+1 | javadoc | 0m 55s | trunk passed |
Patch Compile Tests | |||
+1 | mvninstall | 0m 56s | the patch passed |
+1 | compile | 0m 50s | the patch passed |
+1 | javac | 0m 50s | the patch passed |
-0 | checkstyle | 0m 35s | hadoop-hdfs-project/hadoop-hdfs: The patch generated 14 new + 20 unchanged - 6 fixed = 34 total (was 26) |
+1 | mvnsite | 0m 55s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | shadedclient | 11m 0s | patch has no errors when building and testing our client artifacts. |
+1 | findbugs | 2m 10s | the patch passed |
+1 | javadoc | 0m 51s | the patch passed |
Other Tests | |||
-1 | unit | 86m 18s | hadoop-hdfs in the patch failed. |
+1 | asflicense | 0m 22s | The patch does not generate ASF License warnings. |
137m 46s |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12910309/HDFS-1686.00.patch |
Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
uname | Linux cf61c4ea1920 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18 11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
Build tool | maven |
Personality | /testptch/patchprocess/precommit/personality/provided.sh |
git revision | trunk / 5a1db60 |
maven | version: Apache Maven 3.3.9 |
Default Java | 1.8.0_151 |
findbugs | v3.1.0-RC1 |
checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/23040/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt |
unit | https://builds.apache.org/job/PreCommit-HDFS-Build/23040/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt |
Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/23040/testReport/ |
Max. process+thread count | 4180 (vs. ulimit of 5500) |
modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/23040/console |
Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 26s | Docker mode activated. |
Prechecks | |||
+1 | @author | 0m 1s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
trunk Compile Tests | |||
+1 | mvninstall | 15m 30s | trunk passed |
+1 | compile | 0m 51s | trunk passed |
+1 | checkstyle | 0m 36s | trunk passed |
+1 | mvnsite | 0m 58s | trunk passed |
+1 | shadedclient | 11m 3s | branch has no errors when building and testing our client artifacts. |
+1 | findbugs | 1m 48s | trunk passed |
+1 | javadoc | 0m 54s | trunk passed |
Patch Compile Tests | |||
+1 | mvninstall | 0m 52s | the patch passed |
+1 | compile | 0m 48s | the patch passed |
+1 | javac | 0m 48s | the patch passed |
-0 | checkstyle | 0m 33s | hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 25 unchanged - 1 fixed = 26 total (was 26) |
+1 | mvnsite | 0m 56s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | shadedclient | 10m 37s | patch has no errors when building and testing our client artifacts. |
+1 | findbugs | 1m 56s | the patch passed |
+1 | javadoc | 0m 55s | the patch passed |
Other Tests | |||
-1 | unit | 133m 48s | hadoop-hdfs in the patch failed. |
+1 | asflicense | 0m 23s | The patch does not generate ASF License warnings. |
182m 48s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA |
hadoop.hdfs.web.TestWebHdfsTimeouts | |
hadoop.hdfs.TestDFSStripedOutputStreamWithFailure | |
hadoop.hdfs.TestReadStripedFileWithMissingBlocks |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12910490/HDFS-1686.01.patch |
Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
uname | Linux 387d97cf88a4 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20 11:50:30 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
Build tool | maven |
Personality | /testptch/patchprocess/precommit/personality/provided.sh |
git revision | trunk / 332269d |
maven | version: Apache Maven 3.3.9 |
Default Java | 1.8.0_151 |
findbugs | v3.1.0-RC1 |
checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/23056/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt |
unit | https://builds.apache.org/job/PreCommit-HDFS-Build/23056/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt |
Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/23056/testReport/ |
Max. process+thread count | 4090 (vs. ulimit of 5500) |
modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/23056/console |
Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
- Question: why multiplying replication in computing totalUsed?
- final long totalUsed = totalCapacity*3/10; + final long totalUsed = (totalCapacity * s.replication)*3/10;
- The content of both testBalancingBlockpoolsWithBlockPoolPolicy and test1OutOf2BlockpoolsWithBlockPoolPolicy are the same. Forgot to modify one of them?
Thanks for review.
replication factor is multipled because, when balacer parameters is passed, replication factor two is used.
s = new Suite(cluster, nNameNodes, nDataNodes, params, conf, (short)2);
In One of the test, I want to test both blockPools, I have updated testBalancingBlockpoolsWithBlockPoolPolicy, to pass value 2 for nNameNodestoBalance.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 33s | Docker mode activated. |
Prechecks | |||
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
trunk Compile Tests | |||
+1 | mvninstall | 15m 9s | trunk passed |
+1 | compile | 0m 49s | trunk passed |
+1 | checkstyle | 0m 34s | trunk passed |
+1 | mvnsite | 0m 56s | trunk passed |
+1 | shadedclient | 10m 55s | branch has no errors when building and testing our client artifacts. |
+1 | findbugs | 1m 45s | trunk passed |
+1 | javadoc | 0m 54s | trunk passed |
Patch Compile Tests | |||
+1 | mvninstall | 0m 50s | the patch passed |
+1 | compile | 0m 47s | the patch passed |
+1 | javac | 0m 47s | the patch passed |
+1 | checkstyle | 0m 34s | hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 25 unchanged - 1 fixed = 25 total (was 26) |
+1 | mvnsite | 0m 52s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | shadedclient | 10m 23s | patch has no errors when building and testing our client artifacts. |
+1 | findbugs | 1m 58s | the patch passed |
+1 | javadoc | 0m 51s | the patch passed |
Other Tests | |||
-1 | unit | 146m 7s | hadoop-hdfs in the patch failed. |
+1 | asflicense | 0m 18s | The patch does not generate ASF License warnings. |
194m 10s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA |
hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery | |
hadoop.hdfs.web.TestWebHdfsTimeouts | |
hadoop.hdfs.security.TestDelegationTokenForProxyUser | |
hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting | |
hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean | |
hadoop.hdfs.TestBlocksScheduledCounter |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12912680/HDFS-1686.02.patch |
Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
uname | Linux 126430fb21fe 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20 11:50:30 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
Build tool | maven |
Personality | /testptch/patchprocess/precommit/personality/provided.sh |
git revision | trunk / 96e8f26 |
maven | version: Apache Maven 3.3.9 |
Default Java | 1.8.0_151 |
findbugs | v3.1.0-RC1 |
unit | https://builds.apache.org/job/PreCommit-HDFS-Build/23255/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt |
Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/23255/testReport/ |
Max. process+thread count | 3628 (vs. ulimit of 10000) |
modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/23255/console |
Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13758 (See https://builds.apache.org/job/Hadoop-trunk-Commit/13758/)
HDFS-1686. Federation: Add more Balancer tests with federation setting. (szetszwo: rev 850b2f256722a360d1378b505832cd99c4c5a686)
- (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancerWithMultipleNameNodes.java
4358946.patch: added TestBalancerWith3Nn4Dn2Dn