[IMPALA-10408] Build against Apache official versions

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.5.0
• Component/s: Infrastructure
• Epic Color: ghx-label-5
• Assignee: YUBI LEE
• Reporter: Quanlong Huang
• Created: 28/Dec/20 02:12
• Updated: 28/Aug/24 22:33
• Resolved: 28/Aug/24 22:22

Description

Currently, we are building against CDH/CDP components of Hadoop, Hive, HBase, Tez, Ranger, Sentry etc. The main reason is we need some bug fixes or new features that haven't been released in Apache official versions. Depending on CDH/CDP versions help us evolve faster.

However, building against Apache official releases can increase Impala's adoption on non CDH/CDP users. For released versions like 3.4.0, we can try to find/release compatible Apache official versions for the dependencies.

This would require significant works so we need to make it step by step. There are some discussions in this email thread:
https://lists.apache.org/thread.html/r7bee90b9f0b7c5038798562b2e6340f41d21426cfb8c7978b9ba4842%40%3Cdev.impala.apache.org%3E

Issue Links

• breaks: IMPALA-13311 Hive3 INSERT failed by ClassNotFoundException: org.apache.tez.runtime.api.Event
• breaks: IMPALA-13284 Loading test data on Apache Hive3
• is related to: IMPALA-13327 bin/bootstrap_toolchain.py always download hbase-2.6.0-hadoop3-bin.tar.gz when USE_APACHE_COMPONENTS=true
• relates to: IMPALA-10847 Support on Apache Hive3 Release
• relates to: IMPALA-12349 Support Apache Hive2 in Impala 4.x
• relates to: IMPALA-11454 Reduce the kudu package's size in toolchain