[HBASE-15477] Do not save 'next block header' when we cache hfileblocks

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.3.0, 2.0.0
• Component/s: BlockCache, Performance
• Hadoop Flags: Reviewed

Release Note

Fix over-persisting in blockcache; no longer save the block PLUS the header of the next block (33 bytes) when writing the cache.

Also removes support for hfileblock v1; hfile block v1 was used writing hfile v1. hfile v1 was the default in hbase before hbase-0.92. hbase.96 would not start unless all v1 hfiles had been compacted out of the cluster.

• Assignee: Michael Stack
• Reporter: Michael Stack
• Created: 17/Mar/16 19:43
• Updated: 09/Nov/17 00:48
• Resolved: 26/Apr/16 17:12

Description

When we read from HDFS, we overread to pick up the next blocks header.
Doing this saves a seek as we move through the hfile; we save having to
do an explicit seek just to read the block header every time we need to
read the body. We used to read in the next header as part of the
current blocks buffer. This buffer was then what got persisted to
blockcache; so we were over-persisting wrtiting out our block plus the
next blocks' header (overpersisting 33 bytes). Parse of HFileBlock
complicated by this extra tail. Fix.

Issue Links

• relates to: HBASE-15713 Backport "HBASE-15477 Do not save 'next block header' when we cache hfileblocks"
• links to: Review Board Link