Details
-
Type:
Bug
-
Status: Closed
-
Priority:
Critical
-
Resolution: Fixed
-
Affects Version/s: 2.7.0
-
Fix Version/s: 2.8.0, 2.7.3, 3.0.0-alpha1
-
Component/s: None
-
Labels:None
-
Target Version/s:
-
Hadoop Flags:Reviewed
Description
Unit test TestTextInputFormat.testSplitableCodecs() failed when the seed is 1313094493.
Stacktrace
java.lang.AssertionError: Key in multiple partitions.
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertFalse(Assert.java:64)
at org.apache.hadoop.mapred.TestTextInputFormat.testSplitableCodecs(TestTextInputFormat.java:223)
Issue Links
- is broken by
-
HADOOP-11445
Bzip2Codec: Data block is skipped when position of newly created stream is equal to start of split
-
- Closed
-
Activity
- All
- Comments
- Work Log
- History
- Activity
- Transitions
It seems to be caused by non checking corner case when byte alignment of data and current position of stream is same. In this case BZip2 stream can return same block marker same to previous one.
I fixed BZip2Codec and added above test cases.
 -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 28s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| 0 | mvndep | 0m 11s | Maven dependency ordering for branch |
| +1 | mvninstall | 6m 1s | trunk passed |
| +1 | compile | 6m 11s | trunk passed |
| +1 | checkstyle | 1m 19s | trunk passed |
| +1 | mvnsite | 1m 16s | trunk passed |
| +1 | mvneclipse | 0m 23s | trunk passed |
| +1 | findbugs | 1m 35s | trunk passed |
| +1 | javadoc | 1m 5s | trunk passed |
| 0 | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 | mvninstall | 0m 55s | the patch passed |
| +1 | compile | 6m 9s | the patch passed |
| +1 | javac | 6m 9s | the patch passed |
| -1 | checkstyle | 1m 19s | root: The patch generated 4 new + 52 unchanged - 4 fixed = 56 total (was 56) |
| +1 | mvnsite | 1m 16s | the patch passed |
| +1 | mvneclipse | 0m 23s | the patch passed |
| +1 | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 | findbugs | 1m 52s | the patch passed |
| +1 | javadoc | 1m 3s | the patch passed |
| +1 | unit | 7m 4s | hadoop-common in the patch passed. |
| -1 | unit | 108m 45s | hadoop-mapreduce-client-jobclient in the patch failed. |
| +1 | asflicense | 0m 28s | The patch does not generate ASF License warnings. |
| 148m 42s |
| Reason | Tests |
|---|---|
| Failed junit tests | hadoop.mapred.TestMRCJCFileOutputCommitter |
| hadoop.mapred.TestMiniMRChildTask |
This message was automatically generated.
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 23s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| 0 | mvndep | 0m 22s | Maven dependency ordering for branch |
| +1 | mvninstall | 6m 0s | trunk passed |
| +1 | compile | 6m 10s | trunk passed |
| +1 | checkstyle | 1m 17s | trunk passed |
| +1 | mvnsite | 1m 15s | trunk passed |
| +1 | mvneclipse | 0m 23s | trunk passed |
| +1 | findbugs | 1m 37s | trunk passed |
| +1 | javadoc | 1m 4s | trunk passed |
| 0 | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 | mvninstall | 0m 56s | the patch passed |
| +1 | compile | 6m 11s | the patch passed |
| +1 | javac | 6m 11s | the patch passed |
| +1 | checkstyle | 1m 18s | root: The patch generated 0 new + 52 unchanged - 4 fixed = 52 total (was 56) |
| +1 | mvnsite | 1m 16s | the patch passed |
| +1 | mvneclipse | 0m 23s | the patch passed |
| +1 | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 | findbugs | 1m 52s | the patch passed |
| +1 | javadoc | 1m 5s | the patch passed |
| -1 | unit | 6m 31s | hadoop-common in the patch failed. |
| -1 | unit | 109m 59s | hadoop-mapreduce-client-jobclient in the patch failed. |
| +1 | asflicense | 0m 30s | The patch does not generate ASF License warnings. |
| 149m 29s |
| Reason | Tests |
|---|---|
| Failed junit tests | hadoop.net.TestDNS |
| hadoop.mapred.TestMRCJCFileOutputCommitter | |
| hadoop.mapred.TestMiniMRChildTask |
This message was automatically generated.
Thanks Haibo Chen and Andras Bokor for reporting this and thanks Kai Sasaki for providing the patch.
Hi Kai Sasaki, this patch looks to be reverting HADOOP-11445, so the change would cause HADOOP-11445 again. What do you think?
Akira Ajisaka Thanks you so much for checking related JIRA. As you said, I found the current patch will cause same problem fixed in HADOOP-11445 although a patch in HADOOP-11445 also seems cause this problem.
So looks like we need to investigate further and solve root cause. I'll look into and update the patch.
I found BZip2Codec always back 10bytes to search compression marker even markers except for first marker has 6bytes. It can let BZip2CompressionInputStream find same marker twice because the search start points is back more than marker size. I fixed compression stream only back 6bytes other than the file start point.
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 15s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| 0 | mvndep | 0m 12s | Maven dependency ordering for branch |
| +1 | mvninstall | 6m 23s | trunk passed |
| +1 | compile | 6m 41s | trunk passed |
| +1 | checkstyle | 1m 22s | trunk passed |
| +1 | mvnsite | 1m 23s | trunk passed |
| +1 | mvneclipse | 0m 27s | trunk passed |
| +1 | findbugs | 1m 44s | trunk passed |
| +1 | javadoc | 1m 0s | trunk passed |
| 0 | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 | mvninstall | 1m 0s | the patch passed |
| +1 | compile | 6m 41s | the patch passed |
| +1 | javac | 6m 41s | the patch passed |
| -1 | checkstyle | 1m 20s | root: The patch generated 1 new + 54 unchanged - 2 fixed = 55 total (was 56) |
| +1 | mvnsite | 1m 21s | the patch passed |
| +1 | mvneclipse | 0m 26s | the patch passed |
| -1 | whitespace | 0m 0s | The patch has 20 line(s) that end in whitespace. Use git apply --whitespace=fix. |
| +1 | findbugs | 2m 0s | the patch passed |
| +1 | javadoc | 0m 58s | the patch passed |
| -1 | unit | 21m 30s | hadoop-common in the patch failed. |
| -1 | unit | 109m 37s | hadoop-mapreduce-client-jobclient in the patch failed. |
| +1 | asflicense | 0m 29s | The patch does not generate ASF License warnings. |
| 165m 53s |
| Reason | Tests |
|---|---|
| Failed junit tests | hadoop.mapred.TestMRCJCFileOutputCommitter |
| Timed out junit tests | org.apache.hadoop.http.TestHttpServerLifecycle |
This message was automatically generated.
Thanks Kai Sasaki for updating the patch! Mostly looks good to me. Some comments:
if (start == 0) {
adjStart = Math.max(0L, start - FIRST_BZIP2_BLOCK_MARKER_POSITION);
1. It seems that adjStart is always zero. If that is true, we should set zero instead of using max function.
// Other than the fist of file, the marker size is 6 bytes.
2. fist -> first
3. Would you fix the checkstyle warning?
// corner case when we have byte alignment and position of stream are same
int[] lengths = {471507, 473608};
int[] numSplits = {218, 110};
for (int i = 0; i < 2; i++) {
verifyPartitions(lengths[i], numSplits[i], file, codec, conf);
}
4. (nit) The following code is simpler for me:
// corner case when we have byte alignment and position of stream are same
verifyPartitions(471507, 218, file, codec, conf);
verifyPartitions(473608, 110, file, codec, conf);
Tested the patch against some other seeds with which I have seen the test failed. All green after applying the patch.
Akira Ajisaka Thanks you for reviewing. I updated the patch accordingly.
And also thanks for checking test case Haibo Chen.
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 17s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| 0 | mvndep | 0m 11s | Maven dependency ordering for branch |
| +1 | mvninstall | 6m 11s | trunk passed |
| +1 | compile | 6m 28s | trunk passed |
| +1 | checkstyle | 1m 20s | trunk passed |
| +1 | mvnsite | 1m 19s | trunk passed |
| +1 | mvneclipse | 0m 26s | trunk passed |
| +1 | findbugs | 1m 42s | trunk passed |
| +1 | javadoc | 0m 58s | trunk passed |
| 0 | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 | mvninstall | 0m 57s | the patch passed |
| +1 | compile | 6m 23s | the patch passed |
| +1 | javac | 6m 23s | the patch passed |
| +1 | checkstyle | 1m 19s | root: The patch generated 0 new + 54 unchanged - 2 fixed = 54 total (was 56) |
| +1 | mvnsite | 1m 18s | the patch passed |
| +1 | mvneclipse | 0m 26s | the patch passed |
| -1 | whitespace | 0m 0s | The patch has 20 line(s) that end in whitespace. Use git apply --whitespace=fix. |
| +1 | findbugs | 1m 56s | the patch passed |
| +1 | javadoc | 0m 57s | the patch passed |
| +1 | unit | 6m 59s | hadoop-common in the patch passed. |
| -1 | unit | 104m 42s | hadoop-mapreduce-client-jobclient in the patch failed. |
| +1 | asflicense | 0m 29s | The patch does not generate ASF License warnings. |
| 145m 20s |
| Reason | Tests |
|---|---|
| Failed junit tests | hadoop.mapreduce.TestMRJobClient |
| hadoop.mapred.TestMRCJCFileOutputCommitter |
This message was automatically generated.
whitespace issue seems to be caused by this issue reported in HADOOP-13258.
https://issues.apache.org/jira/browse/HADOOP-13258
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 14s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| 0 | mvndep | 0m 32s | Maven dependency ordering for branch |
| +1 | mvninstall | 6m 21s | trunk passed |
| +1 | compile | 6m 34s | trunk passed |
| +1 | checkstyle | 1m 19s | trunk passed |
| +1 | mvnsite | 1m 19s | trunk passed |
| +1 | mvneclipse | 0m 25s | trunk passed |
| +1 | findbugs | 1m 41s | trunk passed |
| +1 | javadoc | 0m 55s | trunk passed |
| 0 | mvndep | 0m 12s | Maven dependency ordering for patch |
| +1 | mvninstall | 0m 57s | the patch passed |
| +1 | compile | 6m 20s | the patch passed |
| +1 | javac | 6m 20s | the patch passed |
| +1 | checkstyle | 1m 19s | root: The patch generated 0 new + 53 unchanged - 2 fixed = 53 total (was 55) |
| +1 | mvnsite | 1m 16s | the patch passed |
| +1 | mvneclipse | 0m 26s | the patch passed |
| -1 | whitespace | 0m 0s | The patch has 20 line(s) that end in whitespace. Use git apply --whitespace=fix. |
| +1 | findbugs | 1m 57s | the patch passed |
| +1 | javadoc | 0m 56s | the patch passed |
| +1 | unit | 7m 32s | hadoop-common in the patch passed. |
| -1 | unit | 108m 59s | hadoop-mapreduce-client-jobclient in the patch failed. |
| +1 | asflicense | 0m 30s | The patch does not generate ASF License warnings. |
| 150m 34s |
| Reason | Tests |
|---|---|
| Failed junit tests | hadoop.mapred.TestMRCJCFileOutputCommitter |
This message was automatically generated.
+1. Hi Haibo Chen, would you test the latest patch? I'll commit it if you say okay.
Yep, the latest patch fixes all failure cases (seeds) I have.
Moved to Hadoop common and updated the summary because the change is in hadoop-common.
Committed this to branch-2.7 and above. Thanks Kai for the contribution and thanks Haibo for the review.
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 28s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| 0 | mvndep | 0m 17s | Maven dependency ordering for branch |
| +1 | mvninstall | 7m 17s | trunk passed |
| +1 | compile | 7m 38s | trunk passed |
| +1 | checkstyle | 1m 32s | trunk passed |
| +1 | mvnsite | 1m 26s | trunk passed |
| +1 | mvneclipse | 0m 26s | trunk passed |
| +1 | findbugs | 1m 55s | trunk passed |
| +1 | javadoc | 1m 6s | trunk passed |
| 0 | mvndep | 0m 12s | Maven dependency ordering for patch |
| +1 | mvninstall | 1m 4s | the patch passed |
| +1 | compile | 7m 40s | the patch passed |
| +1 | javac | 7m 40s | the patch passed |
| +1 | checkstyle | 1m 32s | root: The patch generated 0 new + 53 unchanged - 2 fixed = 53 total (was 55) |
| +1 | mvnsite | 1m 30s | the patch passed |
| +1 | mvneclipse | 0m 27s | the patch passed |
| -1 | whitespace | 0m 0s | The patch has 20 line(s) that end in whitespace. Use git apply --whitespace=fix. |
| +1 | findbugs | 2m 14s | the patch passed |
| +1 | javadoc | 1m 1s | the patch passed |
| -1 | unit | 16m 59s | hadoop-common in the patch failed. |
| +1 | unit | 114m 33s | hadoop-mapreduce-client-jobclient in the patch passed. |
| +1 | asflicense | 0m 26s | The patch does not generate ASF License warnings. |
| 170m 55s |
| Reason | Tests |
|---|---|
| Failed junit tests | hadoop.metrics2.impl.TestGangliaMetrics |
| Timed out junit tests | org.apache.hadoop.http.TestHttpServerLifecycle |
| Subsystem | Report/Notes |
|---|---|
| Docker | Image:yetus/hadoop:2c91fd8 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12809833/MAPREDUCE-6643.05.patch |
| JIRA Issue | |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle |
| uname | Linux 5408869a6cab 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 709a814 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| whitespace | https://builds.apache.org/job/PreCommit-HADOOP-Build/9764/artifact/patchprocess/whitespace-eol.txt |
| unit | https://builds.apache.org/job/PreCommit-HADOOP-Build/9764/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt |
| Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/9764/testReport/ |
| modules | C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: . |
| Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/9764/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
Closing the JIRA as part of 2.7.3 release.
When seed is 855631073 it also fails.