Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.21.0
    • Component/s: build
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Some tests like TestDFSIO are failing:

      java.lang.NoClassDefFoundError: org/apache/avro/io/DatumReader
      	at java.lang.Class.forName0(Native Method)
      	at java.lang.Class.forName(Class.java:247)
      	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:837)
      	at org.apache.hadoop.io.serializer.SerializationFactory.add(SerializationFactory.java:66)
      	at org.apache.hadoop.io.serializer.SerializationFactory.<init>(SerializationFactory.java:58)
      	at org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:916)
      	at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:852)
      	at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:385)
      	at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:276)
      	at org.apache.hadoop.fs.TestDFSIO.createControlFile(TestDFSIO.java:134)
      	at org.apache.hadoop.fs.TestDFSIO.testIOs(TestDFSIO.java:114)
      	at org.apache.hadoop.fs.TestDFSIO.testIOs(TestDFSIO.java:99)
      Caused by: java.lang.ClassNotFoundException: org.apache.avro.io.DatumReader
      	at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
      	at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
      	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
      	at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
      	at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
      
      1. h534_20090810.patch
        0.5 kB
        Tsz Wo Nicholas Sze
      2. h534_20090810b.patch
        1 kB
        Tsz Wo Nicholas Sze
      3. h534_20090811.patch
        1 kB
        Tsz Wo Nicholas Sze

        Issue Links

          Activity

          Hide
          Tsz Wo Nicholas Sze added a comment -

          h534_20090810.patch: add avro jar to ivy.

          Show
          Tsz Wo Nicholas Sze added a comment - h534_20090810.patch: add avro jar to ivy.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Try hudson.

          Show
          Tsz Wo Nicholas Sze added a comment - Try hudson.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Thanks, Suresh, who pointed out that the previous patch won't work if the arvo jar file is not in the cache.

          h534_20090810b.patch: updated ivysettings.xml as the patch in HADOOP-6120.

          Show
          Tsz Wo Nicholas Sze added a comment - Thanks, Suresh, who pointed out that the previous patch won't work if the arvo jar file is not in the cache. h534_20090810b.patch: updated ivysettings.xml as the patch in HADOOP-6120 .
          Hide
          Suresh Srinivas added a comment -

          +1. I have also tested this change and it fixed many of test failures...

          Show
          Suresh Srinivas added a comment - +1. I have also tested this change and it fixed many of test failures...
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12416128/h534_20090810b.patch
          against trunk revision 802972.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/54/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/54/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/54/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/54/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12416128/h534_20090810b.patch against trunk revision 802972. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/54/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/54/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/54/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/54/console This message is automatically generated.
          Hide
          Tom White added a comment -

          Can you define a variable for the Avro version number in ivysettings.xml and reference it in the "rev" field in ivy.xml? This would be more consistent with the rest of the file.

          Show
          Tom White added a comment - Can you define a variable for the Avro version number in ivysettings.xml and reference it in the "rev" field in ivy.xml? This would be more consistent with the rest of the file.
          Hide
          Tom White added a comment -

          HDFS doesn't depend on Avro directly, Common does, so really we should be using Ivy's transitive dependency support (so that by depending on Common, Ivy will pull in the jars that Common depends on, such as Avro, Jets3t, etc). This requires HADOOP-5107, so the approach in this issue is fine in the meantime.

          Show
          Tom White added a comment - HDFS doesn't depend on Avro directly, Common does, so really we should be using Ivy's transitive dependency support (so that by depending on Common, Ivy will pull in the jars that Common depends on, such as Avro, Jets3t, etc). This requires HADOOP-5107 , so the approach in this issue is fine in the meantime.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          > Can you define a variable for the Avro version number in ivysettings.xml and reference it in the "rev" field in ivy.xml? This would be more consistent with the rest of the file.

          Sure. I guess you mean libraries.properties instead of ivysettings.xml. Thank you for the hints, Tom.

          h534_20090811.patch: define a version variable

          Show
          Tsz Wo Nicholas Sze added a comment - > Can you define a variable for the Avro version number in ivysettings.xml and reference it in the "rev" field in ivy.xml? This would be more consistent with the rest of the file. Sure. I guess you mean libraries.properties instead of ivysettings.xml. Thank you for the hints, Tom. h534_20090811.patch: define a version variable
          Hide
          Tsz Wo Nicholas Sze added a comment -

          > HDFS doesn't depend on Avro directly, Common does, so really we should be using Ivy's transitive dependency support (so that by depending on Common, Ivy will pull in the jars that Common depends on, such as Avro, Jets3t, etc). This requires HADOOP-5107, so the approach in this issue is fine in the meantime.

          Then, I will commit h534_20090810b.patch which has passed Hudson.

          • no new tests since this is an ivy change.
          • TestHDFSTrash is not related.
          Show
          Tsz Wo Nicholas Sze added a comment - > HDFS doesn't depend on Avro directly, Common does, so really we should be using Ivy's transitive dependency support (so that by depending on Common, Ivy will pull in the jars that Common depends on, such as Avro, Jets3t, etc). This requires HADOOP-5107 , so the approach in this issue is fine in the meantime. Then, I will commit h534_20090810b.patch which has passed Hudson. no new tests since this is an ivy change. TestHDFSTrash is not related.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I have committed this. (also updated hadoop-core-0.21.0-dev.jar and hadoop-core-test-0.21.0-dev.jar)

          Show
          Tsz Wo Nicholas Sze added a comment - I have committed this. (also updated hadoop-core-0.21.0-dev.jar and hadoop-core-test-0.21.0-dev.jar)
          Hide
          Tom White added a comment -

          I guess you mean libraries.properties instead of ivysettings.xml.

          Yes, I did.

          Show
          Tom White added a comment - I guess you mean libraries.properties instead of ivysettings.xml. Yes, I did.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #47 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk/47/)
          . Include avro in ivy.

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #47 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk/47/ ) . Include avro in ivy.

            People

            • Assignee:
              Tsz Wo Nicholas Sze
              Reporter:
              Tsz Wo Nicholas Sze
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development