[HIVE-12708] Hive on Spark doesn't work with Kerboresed HBase [Spark Branch]

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.1.0, 1.2.0, 2.0.0
• Fix Version/s: spark-branch, 2.1.0
• Component/s: Spark
• Assignee: Xuefu Zhang
• Reporter: Xuefu Zhang
• Created: 18/Dec/15 18:02
• Updated: 21/Jun/16 15:52
• Resolved: 18/Dec/15 22:37

Description

Spark application launcher (spark-submit) acquires HBase delegation token on Hive user's behalf when the application is launched. This mechanism, which doesn't work for long-running sessions, is not in line with what Hive is doing. Hive actually acquires the token automatically whenever a job needs it. The right approach for Spark should be allowing applications to dynamically add whatever tokens they need to the spark context. While this needs work on Spark side, we provide a workaround solution in Hive.