[KUDU-1396] CFile index builder overflows stack if an individual value is larger than the block size

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.8.0
• Fix Version/s: 0.9.0
• Component/s: cfile
• Target Version/s: 0.9.0
• Code Review: http://gerrit.cloudera.org:8080/#/c/2725/
• Assignee: Todd Lipcon
• Reporter: Todd Lipcon
• Created: 06/Apr/16 06:21
• Updated: 06/Apr/16 21:23
• Resolved: 06/Apr/16 21:23

Description

Darren Hoo ran into this issue on his workload which involves larger cells: if an individual value in a CFile with a value-index is larger than the index block size, it will infinitely recurse upwards adding levels to the index. Each time it adds a value to an index block, it sees that the block is full, and tries to add the index block to (or create) the level above. Of course that level will also fill up, etc.

Amusingly, there is a DCHECK for this assertion that has been there since October 2012, but we lacked test coverage around larger cells.