[HIVE-16490] Hive should not use private HDFS APIs for encryption

• Type: Improvement
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.2.0
• Fix Version/s: 4.0.0-alpha-1
• Component/s: Encryption
• Labels: pull-request-available
• Target Version/s: 3.0.0
• Assignee: Uma Maheswara Rao G
• Reporter: Andrew Wang
• Created: 21/Apr/17 00:13
• Updated: 17/Nov/22 08:55
• Resolved: 22/Jul/20 04:50

Description

When compiling against bleeding edge versions of Hive and Hadoop, we discovered that HIVE-16047 references a private HDFS API, DFSClient, to get at various encryption related information. The private API was recently changed by HADOOP-14104, which broke Hive compilation.

It'd be better to instead use publicly supported APIs. HDFS-11687 has been filed to add whatever encryption APIs are needed by Hive. This JIRA is to move Hive over to these new APIs.

Issue Links

• depends upon: HDFS-11687 Add new public encryption APIs required by Hive
• relates to: HDFS-11689 New exception thrown by DFSClient#isHDFSEncryptionEnabled broke hacky hive code
• relates to: HIVE-16047 Shouldn't try to get KeyProvider unless encryption is enabled
• relates to: HADOOP-14104 Client should always ask namenode for kms provider path.
• links to: GitHub Pull Request #1279
• links to: GitHub Pull Request #1279