[AVRO-1336] Tests TestKeyValueWordCount and TestKeyWordCount fail due to missing files (Trevni mapreduce module)

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.7.4
• Component/s: trevni

Environment

Red Hat Server 6.4
Avro 1.7.4
Maven 3.0.5
Java 1.6 (IBM or Oracle)
Hadoop1 = 1.1.2
Hadoop2 = 2.0.4-alpha

• Release Note: Fix Trevni build with Hadoop 2 in Avro 1.7.4
• Assignee: Rodrigo Trujillo
• Reporter: Rodrigo Trujillo
• Created: 21/May/13 13:02
• Updated: 10/Jan/14 20:10

Description

I tried to build Avro 1.7.4 with Hadoop 2 and get the following errors in Trevni mapreduce module tests:

==================================================
...
Running org.apache.trevni.avro.mapreduce.TestKeyValueWordCount
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.536 sec <<< FAILURE!
Running org.apache.trevni.avro.mapreduce.TestKeyWordCount
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.505 sec <<< FAILURE!
Running org.apache.trevni.avro.TestCases
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.671 sec
Running org.apache.trevni.avro.TestWordCount
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.651 sec

Results :

Tests in error:
testIOFormat(org.apache.trevni.avro.mapreduce.TestKeyValueWordCount): target/wc/trevniMapReduceKeyValueTest/out/part-r-00000/part-0.trv (No such file or directory)
testIOFormat(org.apache.trevni.avro.mapreduce.TestKeyWordCount): target/wc/trevniMapReduceKeyTest/out/part-r-00000/part-0.trv (No such file or directory)

Tests run: 23, Failures: 0, Errors: 2, Skipped: 0

...
==================================================

StackTraces:

-------------------------------------------------------------------------------
Test set: org.apache.trevni.avro.mapreduce.TestKeyValueWordCount
-------------------------------------------------------------------------------
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.537 sec <<< FAILURE!
testIOFormat(org.apache.trevni.avro.mapreduce.TestKeyValueWordCount) Time elapsed: 2.029 sec <<< ERROR!
java.io.FileNotFoundException: target/wc/trevniMapReduceKeyValueTest/out/part-r-00000/part-0.trv (No such file or directory)
at java.io.FileInputStream.<init>(FileInputStream.java:124)
at org.apache.trevni.InputFile.<init>(InputFile.java:33)
at org.apache.trevni.avro.AvroColumnReader$Params.<init>(AvroColumnReader.java:69)
at org.apache.trevni.avro.WordCountUtil.validateCountsFileGenericRecord(WordCountUtil.java:122)
at org.apache.trevni.avro.mapreduce.TestKeyValueWordCount.checkOutputFormat(TestKeyValueWordCount.java:138)
at org.apache.trevni.avro.mapreduce.TestKeyValueWordCount.testIOFormat(TestKeyValueWordCount.java:108)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at java.lang.reflect.Method.invoke(Method.java:611)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at java.lang.reflect.Method.invoke(Method.java:611)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74)
./trevni/avro/target/surefire-reports/org.apache.trevni.avro.mapreduce.TestKeyValueWordCount.txt (END)

-------------------------------------------------------------------------------
Test set: org.apache.trevni.avro.mapreduce.TestKeyWordCount
-------------------------------------------------------------------------------
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.505 sec <<< FAILURE!
testIOFormat(org.apache.trevni.avro.mapreduce.TestKeyWordCount) Time elapsed: 2.039 sec <<< ERROR!
java.io.FileNotFoundException: target/wc/trevniMapReduceKeyTest/out/part-r-00000/part-0.trv (No such file or directory)
at java.io.FileInputStream.<init>(FileInputStream.java:124)
at org.apache.trevni.InputFile.<init>(InputFile.java:33)
at org.apache.trevni.avro.AvroColumnReader$Params.<init>(AvroColumnReader.java:69)
at org.apache.trevni.avro.WordCountUtil.validateCountsFile(WordCountUtil.java:109)
at org.apache.trevni.avro.mapreduce.TestKeyWordCount.checkOutputFormat(TestKeyWordCount.java:149)
at org.apache.trevni.avro.mapreduce.TestKeyWordCount.testIOFormat(TestKeyWordCount.java:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at java.lang.reflect.Method.invoke(Method.java:611)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at java.lang.reflect.Method.invoke(Method.java:611)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74)
./trevni/avro/target/surefire-reports/org.apache.trevni.avro.mapreduce.TestKeyWordCount.txt (END)