[KUDU-1950] TS runs out of FDs for WALs when running many tablets

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.2.0
• Fix Version/s: 1.12.0
• Component/s: consensus, fs, tserver
• Labels: dense-storage scalability
• Assignee: Unassigned
• Reporter: David Alves
• Created: 20/Mar/17 23:57
• Updated: 02/Jun/20 17:44
• Resolved: 02/Jun/20 17:44

Description

A user was recently complaining that his TS ran out of FDs when he had many tablets and that most were being used by WALs.

While this was a bit of a non-usual setup - a lot of un-replicated tablets on a single TS - we're likely to hit this problem as we increase density and try and have many more tablets per tablet server.

The report was that each tablet would use 3 fd's for the WAL. Ultimately we'll likely need some solution akin to the what we've done for the FBM/LBM regarding having an LRU cache of fds for the WAL too.