[SPARK-38909] Encapsulate LevelDB used by ExternalShuffleBlockResolver and YarnShuffleService as LocalDB

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: Spark Core, YARN
• Assignee: Yang Jie
• Reporter: Yang Jie
• Created: 15/Apr/22 02:49
• Updated: 24/Aug/22 13:26
• Resolved: 18/Aug/22 14:59

Description

ExternalShuffleBlockResolver and YarnShuffleService  use LevelDB directly, this is not conducive to extending the use of RocksDB in this scenario. This pr is encapsulated for expansibility. It will be the pre-work of SPARK-38888

Issue Links

• links to: [Github] Pull Request #36200 (LuciferYang)
• links to: [Github] Pull Request #37648 (LuciferYang)
• links to: [Github] Pull Request #37648 (LuciferYang)
• links to: https://github.com/apache/spark/pull/36200