[HDFS-16943] RBF: Implement MySQL based StateStoreDriver

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: hdfs, rbf
• Labels: pull-request-available
• Target Version/s: 3.4.0
• Hadoop Flags: Reviewed
• Assignee: Simbarashe Dzinamarira
• Reporter: Simbarashe Dzinamarira
• Created: 08/Mar/23 00:21
• Updated: 28/Jan/24 07:14
• Resolved: 06/Apr/23 02:42

Description

RBF supports two types of StateStoreDrivers

1. StateStoreFileImpl
2. StateStoreZooKeeperImpl

I propose implementing a third driver that is backed by MySQL.

• StateStoreZooKeeperImpl requires an additional Zookeeper cluster.
• StateStoreFileImpl can use one of the namenodes in the HDFS cluster, but that namenode becomes a single point of failure, introducing coupling between the federated clusters.

 HADOOP-18535 implemented a MySQL token store. When tokens are stored in MySQL, using MySQL for the StateStore as well reduces the number of external dependencies for routers.

Issue Links

• links to: GitHub Pull Request #5469