Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-22097

Incompatible java.util.ArrayList for java 11

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.0.0, 3.1.1
    • Fix Version/s: None
    • Component/s: Query Processor
    • Labels:
      None

      Description

      export JAVA_HOME=/usr/lib/jdk-11.0.3
      export PATH=${JAVA_HOME}/bin:${PATH}
      
      hive> create table t(id int);
      Time taken: 0.035 seconds
      hive> insert into t values(1);
      Query ID = root_20190811155400_7c0e0494-eecb-4c54-a9fd-942ab52a0794
      Total jobs = 3
      Launching Job 1 out of 3
      Number of reduce tasks determined at compile time: 1
      In order to change the average load for a reducer (in bytes):
        set hive.exec.reducers.bytes.per.reducer=<number>
      In order to limit the maximum number of reducers:
        set hive.exec.reducers.max=<number>
      In order to set a constant number of reducers:
        set mapreduce.job.reduces=<number>
      java.lang.RuntimeException: java.lang.NoSuchFieldException: parentOffset
      	at org.apache.hadoop.hive.ql.exec.SerializationUtilities$ArrayListSubListSerializer.<init>(SerializationUtilities.java:390)
      	at org.apache.hadoop.hive.ql.exec.SerializationUtilities$1.create(SerializationUtilities.java:235)
      	at org.apache.hive.com.esotericsoftware.kryo.pool.KryoPoolQueueImpl.borrow(KryoPoolQueueImpl.java:48)
      	at org.apache.hadoop.hive.ql.exec.SerializationUtilities.borrowKryo(SerializationUtilities.java:280)
      	at org.apache.hadoop.hive.ql.exec.Utilities.setBaseWork(Utilities.java:595)
      	at org.apache.hadoop.hive.ql.exec.Utilities.setMapWork(Utilities.java:587)
      	at org.apache.hadoop.hive.ql.exec.Utilities.setMapRedWork(Utilities.java:579)
      	at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:357)
      	at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:159)
      	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:212)
      	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:103)
      	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2317)
      	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1969)
      	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1636)
      	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1396)
      	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1390)
      	at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:162)
      	at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:223)
      	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:242)
      	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:189)
      	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:408)
      	at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:838)
      	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:777)
      	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:696)
      	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
      	at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
      	at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
      Caused by: java.lang.NoSuchFieldException: parentOffset
      	at java.base/java.lang.Class.getDeclaredField(Class.java:2412)
      	at org.apache.hadoop.hive.ql.exec.SerializationUtilities$ArrayListSubListSerializer.<init>(SerializationUtilities.java:384)
      	... 29 more
      Job Submission failed with exception 'java.lang.RuntimeException(java.lang.NoSuchFieldException: parentOffset)'
      FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask. java.lang.NoSuchFieldException: parentOffset
      

      The reason is Java removed parentOffset:

        Attachments

        1. JDK1.8.png
          193 kB
          Yuming Wang
        2. JDK11.png
          121 kB
          Yuming Wang

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                yumwang Yuming Wang
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated: