[ORC-1264] [C++] Add a writer option to align compression block with row group boundary

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: C++
• Assignee: ZhangXin
• Reporter: Gang Wu
• Created: 06/Sep/22 01:57
• Updated: 27/Sep/22 01:56

Description

To reduce unnecessary I/O and decompression when PPD is in effect, we can enforce the compression block to be aligned with the row group boundary. It can help avoid unnecessary I/O and decompression of the filtered row groups before the survived row group within the same compression block. This implementation does not break the format specs and should be transparent to any downstream implementation. The caveat may be worse file size which depends on the data distribution and applied compression algorithm. Therefore we should make it optional and enable it per the user's choice.