[HIVE-18281] HiveServer2 HA for LLAP and Workload Manager

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Component/s: HiveServer2
• Target Version/s: 3.0.0
• Assignee: Prasanth Jayachandran
• Reporter: Prasanth Jayachandran
• Created: 15/Dec/17 03:23
• Updated: 22/May/18 23:13
• Resolved: 16/Mar/18 06:01

Description

When running HS2 with LLAP and Workload Manager, HS2 becomes single point of failure as some of the states for workload management and scheduling are maintained in-memory.
The proposal is to support Active/Passive mode of high availability in which, all HS2 and tez AMs registers with ZooKeeper and a leader have to be chosen which will maintain stateful information. Clients using service discovery will always connect to the leader for submitting queries. There will also be some responsibilities for the leader, failover handling, tez session reconnect etc. Will upload some more detailed information in a separate doc.

Issue Links

• blocks: HIVE-18952 Tez session disconnect and reconnect on HS2 HA failover
• requires: TEZ-3892 getClient API for TezClient
• links to: RB

Sub-Tasks

• Service discovery for Active/Passive HA mode: Closed
• Tez session disconnect and reconnect on HS2 HA failover: Resolved
• Provide a CLI option to manually trigger failover: Closed
• Handle client connections on failover: Closed
• HS2 commands for Active-Passive HA as a replacement for CLI option: Open