Description
A punched hole in the log block manager is exactly the size of a Kudu block, which means there's no guarantee that it's filesystem block aligned. As a result, there's also no guarantee that any space is actually reclaimed.
Consider the pathological case where the filesystem block size is 4096 bytes and each Kudu block is 4095 bytes. Punching holes for five blocks, even if they're contiguous, is going to result in a bunch of zeroing, as each hole is less than a full FS block.
Some ideas for fixing this:
- Round each Kudu block up to the nearest multiple of the FS block size. This should ensure that each Kudu block is FS block aligned and that there's no zeroing, but it comes at the cost of increased disk usage and I/O.
- Make an effort to group together punched holes, perhaps by delaying the hole punching for a little while.
- Accept that these "turds" aren't the common case and handle them on startup, either by aggregating holes and punching them together or by GC'ing entire containers.
- Ditch hole punching in favor of periodic GC.
I'm leaning towards implementing #1 but I thought I'd post the rest in case anyone else has some thoughts on the matter.
Attachments
Issue Links
- relates to
-
KUDU-423 Implement scalable and performant on-disk storage
-
- Open
-