[HIVE-25496] hadoop 3.3.1 / hive 3.2.1 / OpenJDK11 compatible?

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate

Environment

Linux VM

• Assignee: Jerome Le Ray
• Reporter: Jerome Le Ray
• Created: 02/Sep/21 15:11
• Updated: 28/May/22 10:26
• Resolved: 28/May/22 10:26

Description

We used the following configuration
hadoop 3.2.1

hive 3.1.2

PostGres 12

Java - OracleJDK 8

For internal reasons, we have to migrate to OpenJDK11.

So, I've migrated hadoop 3.2.1 to the new version hadoop 3.3.1

When I'm starting the hiveserver2 service, I've got the error :

which: no hbase in (/usr/local/bin:/bin:/usr/pgsql-12/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/jdk-11.0.10+9/bin:/opt/hivemetastore/hadoop-3.3.1/bin:/opt/hivemetastore/apache-hive-3.1.2-bin/b
in)
2021-09-02 16:48:05: Starting HiveServer2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/hivemetastore/hadoop-3.3.1/share/hadoop/common/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hivemetastore/apache-hive-3.1.2-bin/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2021-09-02 16:48:06,744 INFO conf.HiveConf: Found configuration file file:/opt/hivemetastore/apache-hive-3.1.2-bin/conf/hive-site.xml
2021-09-02 16:48:07,169 WARN conf.HiveConf: HiveConf of name hive.metastore.local does not exist
2021-09-02 16:48:07,169 WARN conf.HiveConf: HiveConf of name hive.metastore.thrift.bind.host does not exist
2021-09-02 16:48:07,170 WARN conf.HiveConf: HiveConf of name hive.enforce.bucketing does not exist
2021-09-02 16:48:08,414 INFO server.HiveServer2: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting HiveServer2
STARTUP_MSG: host = lhroelcspt1001.enterprisenet.org/10.90.122.159
STARTUP_MSG: args = [-hiveconf, mapred.job.tracker=local, -hiveconf, fs.default.name=file:///cip-data, -hiveconf, hive.metastore.warehouse.dir=file:////cip-data, --hiveconf, hive.server2.thrif
t.port=10000, --hiveconf, hive.root.logger=INFO,console]
STARTUP_MSG: version = 3.1.2

(...)

STARTUP_MSG: build = git://HW13934/Users/gates/tmp/hive-branch-3.1/hive -r 8190d2be7b7165effa62bd21b7d60ef81fb0e4af; compiled by 'gates' on Thu Aug 22 15:01:18 PDT 2019
************************************************************/
2021-09-02 16:48:08,436 INFO server.HiveServer2: Starting HiveServer2
2021-09-02 16:48:08,462 WARN conf.HiveConf: HiveConf of name hive.metastore.local does not exist
2021-09-02 16:48:08,463 WARN conf.HiveConf: HiveConf of name hive.metastore.thrift.bind.host does not exist
2021-09-02 16:48:08,463 WARN conf.HiveConf: HiveConf of name hive.enforce.bucketing does not exist
Hive Session ID = 440449ff-99b7-429c-82d9-e20bdcc9b46f
2021-09-02 16:48:08,566 INFO SessionState: Hive Session ID = 440449ff-99b7-429c-82d9-e20bdcc9b46f
2021-09-02 16:48:08,566 INFO server.HiveServer2: Shutting down HiveServer2
2021-09-02 16:48:08,584 INFO server.HiveServer2: Stopping/Disconnecting tez sessions.
2021-09-02 16:48:08,585 WARN server.HiveServer2: Error starting HiveServer2 on attempt 1, will retry in 60000ms
java.lang.RuntimeException: Error applying authorization policy on hive configuration: class jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to class java.net.URLClassLoader (jdk.
internal.loader.ClassLoaders$AppClassLoader and java.net.URLClassLoader are in module java.base of loader 'bootstrap')
at org.apache.hive.service.cli.CLIService.init(CLIService.java:118)
at org.apache.hive.service.CompositeService.init(CompositeService.java:59)
at org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:230)
at org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:1036)
at org.apache.hive.service.server.HiveServer2.access$1600(HiveServer2.java:140)
at org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:1305)
at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:1149)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
Caused by: java.lang.ClassCastException: class jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to class java.net.URLClassLoader (jdk.internal.loader.ClassLoaders$AppClassLoader an
d java.net.URLClassLoader are in module java.base of loader 'bootstrap')
at org.apache.hadoop.hive.ql.session.SessionState.<init>(SessionState.java:413)
at org.apache.hadoop.hive.ql.session.SessionState.<init>(SessionState.java:389)
at org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:128)
at org.apache.hive.service.cli.CLIService.init(CLIService.java:115)
... 12 more
2021-09-02 16:49:08,588 INFO server.HiveServer2: Starting HiveServer2

Issue Links

• duplicates: HIVE-24484 Upgrade Hadoop to 3.3.1 And Tez to 0.10.2