Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-4640

Compiling Pig with JDK8 or JDK7 Update 85 breaks Ruby UDFs



    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.17.0
    • Component/s: None
    • Labels:
    • Environment:

      CentOS 6.5
      Pig 0.14.3
      Java 1.7.0 - see below


      If you build pig against java patch version 79, it works, but pig built against 85 does not. Here is the code to reproduce.

      $ java -version
      java version "1.7.0_85"
      OpenJDK Runtime Environment (rhel- u85-b01)
      OpenJDK 64-Bit Server VM (build 24.85-b03, mixed mode)

      Then add this code to `query.pig`. Notice that we're registering a ruby UDF and that's all.

      SET job.priority 'normal'; REGISTER ./utility_udfs.rb using jruby AS utilities;

      Make sure you build pig against this java version.


      Then run the job

      $pig -x local -F query.pig

      You will get this error

      2015-07-24 11:32:25,215 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2999: Unexpected internal error. Native Exception: 'class java.lang.ClassFormatError'; Message: Duplicate method name&signature in class file org/apache/pig/scripting/jruby/RubyDataBag$i$initialize; StackTrace: java.lang.ClassFormatError: Duplicate method name&signature in class file org/apache/pig/scripting/jruby/RubyDataBag$i$initialize
          at java.lang.ClassLoader.defineClass1(Native Method)
          at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
          at org.jruby.util.JRubyClassLoader.defineClass(JRubyClassLoader.java:76)
          at org.jruby.internal.runtime.methods.InvocationMethodFactory.endClass(InvocationMethodFactory.java:1378)
          at org.jruby.internal.runtime.methods.InvocationMethodFactory.getAnnotatedMethodClass(InvocationMethodFactory.java:792)
          at org.jruby.internal.runtime.methods.InvocationMethodFactory.getAnnotatedMethod(InvocationMethodFactory.java:702)
          at org.jruby.RubyModule.defineAnnotatedMethod(RubyModule.java:716)
          at org.jruby.anno.TypePopulator$DefaultTypePopulator.populate(TypePopulator.java:75)
          at org.jruby.RubyModule.defineAnnotatedMethodsIndividually(RubyModule.java:708)
          at org.jruby.RubyModule.defineAnnotatedMethods(RubyModule.java:596)
          at org.apache.pig.scripting.jruby.RubyDataBag.define(RubyDataBag.java:112)
          at org.apache.pig.scripting.jruby.PigJrubyLibrary.load(PigJrubyLibrary.java:76)

      If you're on this Java version, everything runs fine

      $ java -version
      java version "1.7.0_79"
      OpenJDK Runtime Environment (rhel- u79-b14)
      OpenJDK 64-Bit Server VM (build 24.79-b02, mixed mode)

      java 1.7.0_85 also breaks with pig 0.15


          Issue Links



              • Assignee:
                daijy Daniel Dai
                deltreey Edward Winslow
              • Votes:
                0 Vote for this issue
                6 Start watching this issue


                • Created: