[KUDU-1156] Compactions always preferred above flushes when MRS usage <64MB

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Private Beta
• Fix Version/s: 0.5.0
• Component/s: tserver
• Code Review: http://gerrit.sjc.cloudera.com:8080/8213
• Assignee: Jean-Daniel Cryans
• Reporter: Todd Lipcon
• Created: 16/Sep/15 02:18
• Updated: 19/Dec/16 09:03
• Resolved: 19/Sep/15 19:41

Description

Looked a bit at the tablet servers in the YCSB cluster today, after the cluster had hit about 1TB of data each (about 789 tablets per server). YCSB is getting slower and slower, apparently because a lot of RAM is being used up by tables that were written days ago. Looking at the maintenance manager dashboard, we are prioritizing compactions, and never flushing MRS even from tablets that haven't taken an insert in days.

I think once the total number of tablets grows a bit more, we'll be in a situation where we won't flush them, but we're basically out of RAM, and everything will grind to a halt.