[HIVE-23980] Shade guava from existing Hive versions

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.3.7
• Fix Version/s: 2.3.8
• Labels: pull-request-available
• Hadoop Flags: Reviewed
• Assignee: L. C. Hsieh Assign to me
• Reporter: L. C. Hsieh
• Created: 03/Aug/20 19:26
• Updated: 18/Dec/20 20:32
• Resolved: 01/Dec/20 23:22

Description

I'm trying to upgrade Guava version in Spark. The JIRA ticket is SPARK-32502.

Running test hits an error:

sbt.ForkMain$ForkError: sbt.ForkMain$ForkError: java.lang.IllegalAccessError: tried to access method com.google.common.collect.Iterators.emptyIterator()Lcom/google/common/collect/UnmodifiableIterator; from class org.apache.hadoop.hive.ql.exec.FetchOperator
	at org.apache.hadoop.hive.ql.exec.FetchOperator.<init>(FetchOperator.java:108)
	at org.apache.hadoop.hive.ql.exec.FetchTask.initialize(FetchTask.java:87)
	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:541)
	at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)

I know that hive-exec doesn't shade Guava until HIVE-22126 but that work targets 4.0.0. I'm wondering if there is a solution for current Hive versions, e.g. Hive 2.3.7? Any ideas?

Thanks.

Issue Links

• is related to: HIVE-24551 Hive should include transitive dependencies from calcite after shading it
• is related to: HIVE-23998 Upgrave Guava to 27 for Hive 2.3
• relates to: HIVE-24512 Exclude calcite in packaging Hive
• relates to: SPARK-32502 Please fix CVE related to Guava 14.0.1
• relates to: HIVE-22126 hive-exec packaging should shade guava
• links to: GitHub Pull Request #1356
• links to: GitHub Pull Request #1397