Avro
  1. Avro
  2. AVRO-1336

Tests TestKeyValueWordCount and TestKeyWordCount fail due to missing files (Trevni mapreduce module)

    Details

    • Type: Bug Bug
    • Status: Patch Available
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 1.7.4
    • Fix Version/s: None
    • Component/s: trevni
    • Labels:
      None
    • Environment:

      Red Hat Server 6.4
      Avro 1.7.4
      Maven 3.0.5
      Java 1.6 (IBM or Oracle)
      Hadoop1 = 1.1.2
      Hadoop2 = 2.0.4-alpha

    • Release Note:
      Fix Trevni build with Hadoop 2 in Avro 1.7.4

      Description

      I tried to build Avro 1.7.4 with Hadoop 2 and get the following errors in Trevni mapreduce module tests:

      ==================================================
      ...
      Running org.apache.trevni.avro.mapreduce.TestKeyValueWordCount
      Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.536 sec <<< FAILURE!
      Running org.apache.trevni.avro.mapreduce.TestKeyWordCount
      Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.505 sec <<< FAILURE!
      Running org.apache.trevni.avro.TestCases
      Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.671 sec
      Running org.apache.trevni.avro.TestWordCount
      Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.651 sec

      Results :

      Tests in error:
      testIOFormat(org.apache.trevni.avro.mapreduce.TestKeyValueWordCount): target/wc/trevniMapReduceKeyValueTest/out/part-r-00000/part-0.trv (No such file or directory)
      testIOFormat(org.apache.trevni.avro.mapreduce.TestKeyWordCount): target/wc/trevniMapReduceKeyTest/out/part-r-00000/part-0.trv (No such file or directory)

      Tests run: 23, Failures: 0, Errors: 2, Skipped: 0

      ...
      ==================================================

      StackTraces:

      -------------------------------------------------------------------------------
      Test set: org.apache.trevni.avro.mapreduce.TestKeyValueWordCount
      -------------------------------------------------------------------------------
      Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.537 sec <<< FAILURE!
      testIOFormat(org.apache.trevni.avro.mapreduce.TestKeyValueWordCount) Time elapsed: 2.029 sec <<< ERROR!
      java.io.FileNotFoundException: target/wc/trevniMapReduceKeyValueTest/out/part-r-00000/part-0.trv (No such file or directory)
      at java.io.FileInputStream.<init>(FileInputStream.java:124)
      at org.apache.trevni.InputFile.<init>(InputFile.java:33)
      at org.apache.trevni.avro.AvroColumnReader$Params.<init>(AvroColumnReader.java:69)
      at org.apache.trevni.avro.WordCountUtil.validateCountsFileGenericRecord(WordCountUtil.java:122)
      at org.apache.trevni.avro.mapreduce.TestKeyValueWordCount.checkOutputFormat(TestKeyValueWordCount.java:138)
      at org.apache.trevni.avro.mapreduce.TestKeyValueWordCount.testIOFormat(TestKeyValueWordCount.java:108)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
      at java.lang.reflect.Method.invoke(Method.java:611)
      at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
      at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
      at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
      at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
      at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
      at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
      at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
      at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
      at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
      at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
      at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
      at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
      at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
      at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236)
      at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134)
      at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
      at java.lang.reflect.Method.invoke(Method.java:611)
      at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
      at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
      at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
      at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103)
      at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74)
      ./trevni/avro/target/surefire-reports/org.apache.trevni.avro.mapreduce.TestKeyValueWordCount.txt (END)

      -------------------------------------------------------------------------------
      Test set: org.apache.trevni.avro.mapreduce.TestKeyWordCount
      -------------------------------------------------------------------------------
      Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.505 sec <<< FAILURE!
      testIOFormat(org.apache.trevni.avro.mapreduce.TestKeyWordCount) Time elapsed: 2.039 sec <<< ERROR!
      java.io.FileNotFoundException: target/wc/trevniMapReduceKeyTest/out/part-r-00000/part-0.trv (No such file or directory)
      at java.io.FileInputStream.<init>(FileInputStream.java:124)
      at org.apache.trevni.InputFile.<init>(InputFile.java:33)
      at org.apache.trevni.avro.AvroColumnReader$Params.<init>(AvroColumnReader.java:69)
      at org.apache.trevni.avro.WordCountUtil.validateCountsFile(WordCountUtil.java:109)
      at org.apache.trevni.avro.mapreduce.TestKeyWordCount.checkOutputFormat(TestKeyWordCount.java:149)
      at org.apache.trevni.avro.mapreduce.TestKeyWordCount.testIOFormat(TestKeyWordCount.java:120)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
      at java.lang.reflect.Method.invoke(Method.java:611)
      at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
      at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
      at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
      at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
      at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
      at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
      at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
      at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
      at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
      at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
      at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
      at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
      at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
      at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236)
      at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134)
      at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
      at java.lang.reflect.Method.invoke(Method.java:611)
      at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
      at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
      at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
      at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103)
      at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74)
      ./trevni/avro/target/surefire-reports/org.apache.trevni.avro.mapreduce.TestKeyWordCount.txt (END)

        Activity

        Rodrigo Trujillo created issue -
        Rodrigo Trujillo made changes -
        Field Original Value New Value
        Assignee Rodrigo Trujillo [ rodrigotrujillo ]
        Rodrigo Trujillo made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Release Note Fix Trevni build with Hadoop 2 in Avro 1.7.4
        Fix Version/s 1.7.4 [ 12323742 ]
        Rodrigo Trujillo made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Rodrigo Trujillo made changes -
        Status Open [ 1 ] In Progress [ 3 ]
        Rodrigo Trujillo made changes -
        Status In Progress [ 3 ] Patch Available [ 10002 ]
        Rodrigo Trujillo made changes -
        Doug Cutting made changes -
        Fix Version/s 1.7.6 [ 12324942 ]
        Fix Version/s 1.7.4 [ 12323742 ]
        Doug Cutting made changes -
        Fix Version/s 1.7.6 [ 12324942 ]

          People

          • Assignee:
            Rodrigo Trujillo
            Reporter:
            Rodrigo Trujillo
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:

              Development