[KUDU-701] blocks can be deleted before associated metadata is flushed

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Private Beta
• Component/s: tablet
• Target Version/s: Feature Complete
• Code Review: http://gerrit.sjc.cloudera.com:8080/#/c/7093/
• Assignee: Todd Lipcon
• Reporter: Todd Lipcon
• Created: 09/Apr/15 18:12
• Updated: 09/Jul/15 19:23
• Resolved: 09/Jul/15 19:23

Description

On the ITBLL cluster we experienced a data loss where several rowsets worth of data blocks went missing. This caused the tablet to fail to bootstrap when the TS was restarted.

After log-spelunking it turns out there is a subtle race when multiple concurrent compactions to a tablet complete at exactly the same time, and then the server crashes.