[ORC-91] Use hdfs v-blocks instead of zero-padding stripes

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.5.0
• Assignee: Gopal Vijayaraghavan
• Reporter: Gopal Vijayaraghavan
• Created: 05/Aug/16 23:45
• Updated: 14/May/18 22:24
• Resolved: 07/Sep/17 20:37

Description

HDFS-3689 added variable length blocks to HDFS as a core feature.

hsync(SyncFlag.END_BLOCK)

can now end an HDFS block on disk without padding it out with zeros.

The current space wasted in ORC to padding is 5%.

Issue Links

• links to: GitHub Pull Request #138
• links to: GitHub Pull Request #166
• links to: GitHub Pull Request #167