Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.21.0
    • Component/s: build
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Some tests like TestDFSIO are failing:

      java.lang.NoClassDefFoundError: org/apache/avro/io/DatumReader
      	at java.lang.Class.forName0(Native Method)
      	at java.lang.Class.forName(Class.java:247)
      	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:837)
      	at org.apache.hadoop.io.serializer.SerializationFactory.add(SerializationFactory.java:66)
      	at org.apache.hadoop.io.serializer.SerializationFactory.<init>(SerializationFactory.java:58)
      	at org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:916)
      	at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:852)
      	at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:385)
      	at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:276)
      	at org.apache.hadoop.fs.TestDFSIO.createControlFile(TestDFSIO.java:134)
      	at org.apache.hadoop.fs.TestDFSIO.testIOs(TestDFSIO.java:114)
      	at org.apache.hadoop.fs.TestDFSIO.testIOs(TestDFSIO.java:99)
      Caused by: java.lang.ClassNotFoundException: org.apache.avro.io.DatumReader
      	at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
      	at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
      	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
      	at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
      	at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
      
      1. h534_20090810.patch
        0.5 kB
        Tsz Wo Nicholas Sze
      2. h534_20090810b.patch
        1 kB
        Tsz Wo Nicholas Sze
      3. h534_20090811.patch
        1 kB
        Tsz Wo Nicholas Sze

        Issue Links

          Activity

          Tsz Wo Nicholas Sze created issue -
          Tsz Wo Nicholas Sze made changes -
          Field Original Value New Value
          Link This issue is related to HADOOP-6120 [ HADOOP-6120 ]
          Hide
          Tsz Wo Nicholas Sze added a comment -

          h534_20090810.patch: add avro jar to ivy.

          Show
          Tsz Wo Nicholas Sze added a comment - h534_20090810.patch: add avro jar to ivy.
          Tsz Wo Nicholas Sze made changes -
          Attachment h534_20090810.patch [ 12416111 ]
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Try hudson.

          Show
          Tsz Wo Nicholas Sze added a comment - Try hudson.
          Tsz Wo Nicholas Sze made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Affects Version/s 0.21.0 [ 12314046 ]
          Assignee Tsz Wo (Nicholas), SZE [ szetszwo ]
          Fix Version/s 0.21.0 [ 12314046 ]
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Thanks, Suresh, who pointed out that the previous patch won't work if the arvo jar file is not in the cache.

          h534_20090810b.patch: updated ivysettings.xml as the patch in HADOOP-6120.

          Show
          Tsz Wo Nicholas Sze added a comment - Thanks, Suresh, who pointed out that the previous patch won't work if the arvo jar file is not in the cache. h534_20090810b.patch: updated ivysettings.xml as the patch in HADOOP-6120 .
          Tsz Wo Nicholas Sze made changes -
          Attachment h534_20090810b.patch [ 12416128 ]
          Hide
          Suresh Srinivas added a comment -

          +1. I have also tested this change and it fixed many of test failures...

          Show
          Suresh Srinivas added a comment - +1. I have also tested this change and it fixed many of test failures...
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12416128/h534_20090810b.patch
          against trunk revision 802972.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/54/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/54/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/54/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/54/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12416128/h534_20090810b.patch against trunk revision 802972. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/54/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/54/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/54/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/54/console This message is automatically generated.
          Hide
          Tom White added a comment -

          Can you define a variable for the Avro version number in ivysettings.xml and reference it in the "rev" field in ivy.xml? This would be more consistent with the rest of the file.

          Show
          Tom White added a comment - Can you define a variable for the Avro version number in ivysettings.xml and reference it in the "rev" field in ivy.xml? This would be more consistent with the rest of the file.
          Hide
          Tom White added a comment -

          HDFS doesn't depend on Avro directly, Common does, so really we should be using Ivy's transitive dependency support (so that by depending on Common, Ivy will pull in the jars that Common depends on, such as Avro, Jets3t, etc). This requires HADOOP-5107, so the approach in this issue is fine in the meantime.

          Show
          Tom White added a comment - HDFS doesn't depend on Avro directly, Common does, so really we should be using Ivy's transitive dependency support (so that by depending on Common, Ivy will pull in the jars that Common depends on, such as Avro, Jets3t, etc). This requires HADOOP-5107 , so the approach in this issue is fine in the meantime.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          > Can you define a variable for the Avro version number in ivysettings.xml and reference it in the "rev" field in ivy.xml? This would be more consistent with the rest of the file.

          Sure. I guess you mean libraries.properties instead of ivysettings.xml. Thank you for the hints, Tom.

          h534_20090811.patch: define a version variable

          Show
          Tsz Wo Nicholas Sze added a comment - > Can you define a variable for the Avro version number in ivysettings.xml and reference it in the "rev" field in ivy.xml? This would be more consistent with the rest of the file. Sure. I guess you mean libraries.properties instead of ivysettings.xml. Thank you for the hints, Tom. h534_20090811.patch: define a version variable
          Tsz Wo Nicholas Sze made changes -
          Attachment h534_20090811.patch [ 12416247 ]
          Hide
          Tsz Wo Nicholas Sze added a comment -

          > HDFS doesn't depend on Avro directly, Common does, so really we should be using Ivy's transitive dependency support (so that by depending on Common, Ivy will pull in the jars that Common depends on, such as Avro, Jets3t, etc). This requires HADOOP-5107, so the approach in this issue is fine in the meantime.

          Then, I will commit h534_20090810b.patch which has passed Hudson.

          • no new tests since this is an ivy change.
          • TestHDFSTrash is not related.
          Show
          Tsz Wo Nicholas Sze added a comment - > HDFS doesn't depend on Avro directly, Common does, so really we should be using Ivy's transitive dependency support (so that by depending on Common, Ivy will pull in the jars that Common depends on, such as Avro, Jets3t, etc). This requires HADOOP-5107 , so the approach in this issue is fine in the meantime. Then, I will commit h534_20090810b.patch which has passed Hudson. no new tests since this is an ivy change. TestHDFSTrash is not related.
          Tsz Wo Nicholas Sze made changes -
          Hadoop Flags [Reviewed]
          Component/s build [ 12312925 ]
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I have committed this. (also updated hadoop-core-0.21.0-dev.jar and hadoop-core-test-0.21.0-dev.jar)

          Show
          Tsz Wo Nicholas Sze added a comment - I have committed this. (also updated hadoop-core-0.21.0-dev.jar and hadoop-core-test-0.21.0-dev.jar)
          Tsz Wo Nicholas Sze made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Tom White added a comment -

          I guess you mean libraries.properties instead of ivysettings.xml.

          Yes, I did.

          Show
          Tom White added a comment - I guess you mean libraries.properties instead of ivysettings.xml. Yes, I did.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #47 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk/47/)
          . Include avro in ivy.

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #47 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk/47/ ) . Include avro in ivy.
          Tom White made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          1h 38m 1 Tsz Wo Nicholas Sze 10/Aug/09 21:45
          Patch Available Patch Available Resolved Resolved
          1d 4h 9m 1 Tsz Wo Nicholas Sze 12/Aug/09 01:54
          Resolved Resolved Closed Closed
          377d 19h 54m 1 Tom White 24/Aug/10 21:48

            People

            • Assignee:
              Tsz Wo Nicholas Sze
              Reporter:
              Tsz Wo Nicholas Sze
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development