[PIG-4640] Compiling Pig with JDK8 or JDK7 Update 85 breaks Ruby UDFs

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.17.0

Environment

CentOS 6.5
Pig 0.14.3
Java 1.7.0 - see below

• Assignee: Daniel Dai
• Reporter: Edward Winslow
• Created: 24/Jul/15 16:51
• Updated: 21/Jun/17 09:15
• Resolved: 01/Apr/17 04:59

Description

If you build pig against java patch version 79, it works, but pig built against 85 does not. Here is the code to reproduce.

$ java -version
java version "1.7.0_85"
OpenJDK Runtime Environment (rhel-2.6.1.3.el6_6-x86_64 u85-b01)
OpenJDK 64-Bit Server VM (build 24.85-b03, mixed mode)

Then add this code to `query.pig`. Notice that we're registering a ruby UDF and that's all.

SET job.priority 'normal'; REGISTER ./utility_udfs.rb using jruby AS utilities;

Make sure you build pig against this java version.

$ant

Then run the job

$pig -x local -F query.pig

You will get this error

2015-07-24 11:32:25,215 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2999: Unexpected internal error. Native Exception: 'class java.lang.ClassFormatError'; Message: Duplicate method name&signature in class file org/apache/pig/scripting/jruby/RubyDataBag$i$initialize; StackTrace: java.lang.ClassFormatError: Duplicate method name&signature in class file org/apache/pig/scripting/jruby/RubyDataBag$i$initialize
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
    at org.jruby.util.JRubyClassLoader.defineClass(JRubyClassLoader.java:76)
    at org.jruby.internal.runtime.methods.InvocationMethodFactory.endClass(InvocationMethodFactory.java:1378)
    at org.jruby.internal.runtime.methods.InvocationMethodFactory.getAnnotatedMethodClass(InvocationMethodFactory.java:792)
    at org.jruby.internal.runtime.methods.InvocationMethodFactory.getAnnotatedMethod(InvocationMethodFactory.java:702)
    at org.jruby.RubyModule.defineAnnotatedMethod(RubyModule.java:716)
    at org.jruby.anno.TypePopulator$DefaultTypePopulator.populate(TypePopulator.java:75)
    at org.jruby.RubyModule.defineAnnotatedMethodsIndividually(RubyModule.java:708)
    at org.jruby.RubyModule.defineAnnotatedMethods(RubyModule.java:596)
    at org.apache.pig.scripting.jruby.RubyDataBag.define(RubyDataBag.java:112)
    at org.apache.pig.scripting.jruby.PigJrubyLibrary.load(PigJrubyLibrary.java:76)

If you're on this Java version, everything runs fine

$ java -version
java version "1.7.0_79"
OpenJDK Runtime Environment (rhel-2.5.5.3.el6_6-x86_64 u79-b14)
OpenJDK 64-Bit Server VM (build 24.79-b02, mixed mode)

java 1.7.0_85 also breaks with pig 0.15

Issue Links

• is part of: PIG-5175 Upgrade jruby to 1.7.26
• relates to: PIG-5175 Upgrade jruby to 1.7.26