[HBASE-6799] Store more metadata in HFiles

• Type: Brainstorming
• Status: Closed
• Priority: Major
• Resolution: Won't Fix
• Assignee: Unassigned
• Reporter: Lars Hofhansl
• Created: 17/Sep/12 04:38
• Updated: 14/Jun/22 18:00
• Resolved: 19/Apr/13 21:01

Description

Current we store metadata in HFile:

• the timerange of KVs
• the earliest PUT ts
• max sequence id
• whether or not this file was created from a major compaction.

I would like to brainstorm what extra data we need to store to make an HFile self describing. I.e. it could be backed up to somewhere with external tools (without invoking an HBase server) can gleam enough information from it to make use of the data inside. Ideally it would also be nice to be able to recreate .META. from a bunch of HFiles to standup a temporary HBase instance to process a bunch of HFiles.

What I can think of:

• min/max key
• table
• column family (or families to be future proof)
• custom tags (set by a backup tools for example)

Issue Links

• relates to: HBASE-6572 Tiered HFile storage