Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-6799

Store more metadata in HFiles

    XMLWordPrintableJSON

    Details

    • Type: Brainstorming
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Current we store metadata in HFile:

      • the timerange of KVs
      • the earliest PUT ts
      • max sequence id
      • whether or not this file was created from a major compaction.

      I would like to brainstorm what extra data we need to store to make an HFile self describing. I.e. it could be backed up to somewhere with external tools (without invoking an HBase server) can gleam enough information from it to make use of the data inside. Ideally it would also be nice to be able to recreate .META. from a bunch of HFiles to standup a temporary HBase instance to process a bunch of HFiles.

      What I can think of:

      • min/max key
      • table
      • column family (or families to be future proof)
      • custom tags (set by a backup tools for example)

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                larsh Lars Hofhansl
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: