Uploaded image for project: 'Avro'
  1. Avro
  2. AVRO-1336

Tests TestKeyValueWordCount and TestKeyWordCount fail due to missing files (Trevni mapreduce module)

    Details

    • Type: Bug
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.7.4
    • Fix Version/s: None
    • Component/s: trevni
    • Labels:
      None
    • Environment:

      Red Hat Server 6.4
      Avro 1.7.4
      Maven 3.0.5
      Java 1.6 (IBM or Oracle)
      Hadoop1 = 1.1.2
      Hadoop2 = 2.0.4-alpha

    • Release Note:
      Fix Trevni build with Hadoop 2 in Avro 1.7.4

      Description

      I tried to build Avro 1.7.4 with Hadoop 2 and get the following errors in Trevni mapreduce module tests:

      ==================================================
      ...
      Running org.apache.trevni.avro.mapreduce.TestKeyValueWordCount
      Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.536 sec <<< FAILURE!
      Running org.apache.trevni.avro.mapreduce.TestKeyWordCount
      Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.505 sec <<< FAILURE!
      Running org.apache.trevni.avro.TestCases
      Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.671 sec
      Running org.apache.trevni.avro.TestWordCount
      Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.651 sec

      Results :

      Tests in error:
      testIOFormat(org.apache.trevni.avro.mapreduce.TestKeyValueWordCount): target/wc/trevniMapReduceKeyValueTest/out/part-r-00000/part-0.trv (No such file or directory)
      testIOFormat(org.apache.trevni.avro.mapreduce.TestKeyWordCount): target/wc/trevniMapReduceKeyTest/out/part-r-00000/part-0.trv (No such file or directory)

      Tests run: 23, Failures: 0, Errors: 2, Skipped: 0

      ...
      ==================================================

      StackTraces:

      -------------------------------------------------------------------------------
      Test set: org.apache.trevni.avro.mapreduce.TestKeyValueWordCount
      -------------------------------------------------------------------------------
      Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.537 sec <<< FAILURE!
      testIOFormat(org.apache.trevni.avro.mapreduce.TestKeyValueWordCount) Time elapsed: 2.029 sec <<< ERROR!
      java.io.FileNotFoundException: target/wc/trevniMapReduceKeyValueTest/out/part-r-00000/part-0.trv (No such file or directory)
      at java.io.FileInputStream.<init>(FileInputStream.java:124)
      at org.apache.trevni.InputFile.<init>(InputFile.java:33)
      at org.apache.trevni.avro.AvroColumnReader$Params.<init>(AvroColumnReader.java:69)
      at org.apache.trevni.avro.WordCountUtil.validateCountsFileGenericRecord(WordCountUtil.java:122)
      at org.apache.trevni.avro.mapreduce.TestKeyValueWordCount.checkOutputFormat(TestKeyValueWordCount.java:138)
      at org.apache.trevni.avro.mapreduce.TestKeyValueWordCount.testIOFormat(TestKeyValueWordCount.java:108)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
      at java.lang.reflect.Method.invoke(Method.java:611)
      at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
      at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
      at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
      at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
      at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
      at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
      at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
      at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
      at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
      at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
      at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
      at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
      at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
      at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236)
      at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134)
      at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
      at java.lang.reflect.Method.invoke(Method.java:611)
      at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
      at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
      at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
      at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103)
      at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74)
      ./trevni/avro/target/surefire-reports/org.apache.trevni.avro.mapreduce.TestKeyValueWordCount.txt (END)

      -------------------------------------------------------------------------------
      Test set: org.apache.trevni.avro.mapreduce.TestKeyWordCount
      -------------------------------------------------------------------------------
      Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.505 sec <<< FAILURE!
      testIOFormat(org.apache.trevni.avro.mapreduce.TestKeyWordCount) Time elapsed: 2.039 sec <<< ERROR!
      java.io.FileNotFoundException: target/wc/trevniMapReduceKeyTest/out/part-r-00000/part-0.trv (No such file or directory)
      at java.io.FileInputStream.<init>(FileInputStream.java:124)
      at org.apache.trevni.InputFile.<init>(InputFile.java:33)
      at org.apache.trevni.avro.AvroColumnReader$Params.<init>(AvroColumnReader.java:69)
      at org.apache.trevni.avro.WordCountUtil.validateCountsFile(WordCountUtil.java:109)
      at org.apache.trevni.avro.mapreduce.TestKeyWordCount.checkOutputFormat(TestKeyWordCount.java:149)
      at org.apache.trevni.avro.mapreduce.TestKeyWordCount.testIOFormat(TestKeyWordCount.java:120)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
      at java.lang.reflect.Method.invoke(Method.java:611)
      at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
      at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
      at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
      at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
      at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
      at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
      at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
      at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
      at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
      at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
      at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
      at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
      at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
      at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236)
      at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134)
      at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
      at java.lang.reflect.Method.invoke(Method.java:611)
      at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
      at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
      at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
      at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103)
      at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74)
      ./trevni/avro/target/surefire-reports/org.apache.trevni.avro.mapreduce.TestKeyWordCount.txt (END)

        Attachments

          Activity

            People

            • Assignee:
              rodrigotrujillo Rodrigo Trujillo
              Reporter:
              rodrigotrujillo Rodrigo Trujillo
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: