[KUDU-2117] Spread metadata across multiple data directories

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.6.0
• Component/s: fs, tablet, tserver
• Labels: scalability
• Assignee: Andrew Wong
• Reporter: Andrew Wong
• Created: 28/Aug/17 18:07
• Updated: 02/Jun/20 20:34

Description

Tablet metadata and consensus metadata are placed in the first configured data directory. This is an issue, as every write to these metadata incurs an fsync and this stresses the first disk considerably more than the others

One way around this is to spread metadata across multiple data directories. A natural choice would be to place them in a directory within the tablet's disk group. In this way, the data/metadata can be completely localized to the disk group, which has the added benefit of making the risk of disk failures easier to assess per tablet.

Issue Links

• is related to: KUDU-2975 Spread WAL across multiple data directories
• is related to: KUDU-1489 Use WAL directory for tablet metadata files
• relates to: KUDU-2204 Revamp storage of tablet server metadata
• relates to: KUDU-2135 Persist disk health state to disk