Uploaded image for project: 'Apache Avro'
  1. Apache Avro
  2. AVRO-1567

Avro java tools tests fail with IBM JVM

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 1.7.4, 1.7.7
    • 1.8.0
    • java
    • None
    • RHEL 6.5 on x86_64
      IBM JVM 7.1.1.1
      HADOOP 2.4.1

    Description

      When using IBM JVM, compared to Oracle JVM, 25 of the Avro Tools tests fail.

      This is due to Avro using Hadoop which uses class:
      org/apache/hadoop/security/UserGroupInformation.java
      which makes use of:
      com.sun.security.auth.module.UnixLoginModule
      which does not exist in IBM JVM.
      Instead there is the class:
      com.ibm.security.auth.module.LinuxLoginModule
      that can be used in UserGroupInformation.java if the JVM is IBM.

      With a IBM-JVM patched version of Hadoop that takes care of the kind of JVM, these 25 Avro Java Tools tests still fail because the pom.xml file of:
      lang/java/tools/
      says unconditionnaly (starting line 146 in Avro 1.7.7) :
      <dependency>
      <groupId>org.apache.hadoop</groupId>
      <artifactId>hadoop-core</artifactId>

      Using:
      mvn -Phadoop2 -Dhadoop.version=2 test
      is of no help.

      In fact, hadoop-core exists only for old Hadoop versions (here version 0.20.205.0 is used by Avro), and not for Hadoop 2.4.1 .

      Replacing hadoop-core by hadoop-client in lang/java/tools/pom.xml file does fix the issue, as a work-around.

      However, a more rigorous solution is required, like it is done in lang/java/mapred/pom.xm , where hadoop-core is associated with hadoop1 and hadoop-client is associated with hadoop2 .

      I'm not an expert of Maven/pom.xml, and the pom.xml file of tools contains <exclusions> tags I have no idea. So, I'm not sure I can provide a correct patch.

      I guess that a Maven/pom.xml expert should be able to fix this in some minutes, plus testing.

      Attachments

        Activity

          People

            Unassigned Unassigned
            trex58 Tony Reix
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - 1h
                1h
                Remaining:
                Remaining Estimate - 1h
                1h
                Logged:
                Time Spent - Not Specified
                Not Specified