[IMPALA-8766] Change hadoop cloud dependencies to use hadoop-cloud-storage

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 3.3.0
• Fix Version/s: Impala 3.3.0
• Component/s: Infrastructure
• Target Version: Impala 3.3.0
• Epic Color: ghx-label-1
• Assignee: Joe McDonnell
• Reporter: Joe McDonnell
• Created: 16/Jul/19 20:04
• Updated: 13/Aug/19 03:12
• Resolved: 27/Jul/19 21:45

Description

Currently, fe/pom.xml specifically includes hadoop-aws, hadoop-azure, and hadoop-azure-datalake directly. There is a meta-package in hadoop called hadoop-cloud-storage that includes these dependencies and others as customized by the hadoop provider, with appropriate exclusions applied to each package.

Migrating Impala to use this meta-package would make it easier for different providers of hadoop to customize hadoop-cloud-storage and the resulting CLASSPATH without needing to change Impala. For example, a hadoop provider may want to include Apache Knox for cloud identity management.