[KUDU-2975] Spread WAL across multiple data directories

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: fs, perf, tablet, tserver
• Labels: performance roadmap-candidate scalability
• Code Review: https://gerrit.cloudera.org/c/14920/
• Assignee: YangSong
• Reporter: LiFu He
• Created: 16/Oct/19 08:03
• Updated: 02/Mar/21 05:54

Description

Recently, we deployed a new kudu cluster and every node has 12 SSD. Then, we created a big table and loaded data to it through flink.  We noticed that the util of one SSD which is used to store WAL is 100% but others are free. So, we suggest to spread WAL across multiple data directories.

Issue Links

• relates to: KUDU-3252 FsManager initialized contention if two tserver instances try to call CreateInitialFileSystemLayout on the same folder
• relates to: KUDU-2117 Spread metadata across multiple data directories