[KNOX-1041] High Availability Support For Apache SOLR, HBase & Kafka

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.12.0, 0.13.0
• Fix Version/s: 0.14.0
• Component/s: Server
• Assignee: Richard Kellog Jr.
• Reporter: Richard Kellog Jr.
• Created: 14/Sep/17 15:19
• Updated: 28/Mar/19 14:07
• Resolved: 20/Oct/17 14:40

Description

Provide high-availability/fail-over between Knox and SOLR/HBase/Kafka using the existing DefaultHaDispatch mechanism and a customized URLManager implementation with knowledge of active hosts in Zookeeper.

When SOLR Cloud is used the active hosts are stored in Zookeeper under the /live_nodes path. The attached custom URLManager implementation queries Zookeeper for the active hosts upon startup. In the event of fail-over, it updates the internal list of hosts.

The HS2ZookeeperURLManager implementation used to provide similar functionality for Hive was used as a starting point.

Issue Links

• is related to: HBASE-3119 [rest] publish endpoint and statistics into ZooKeeper
• relates to: KNOX-587 Implement a provider for Apache ZooKeeper
• relates to: KNOX-567 Having Knox Supporting Hadoop/Yarn/HBase/Hive High Availability
• relates to: KNOX-1053 Document Support for Apache SOLR + HA Support for HBase & Kafka
• links to: Register active Kafka REST Proxy Servers with ZooKeeper