Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-3856

Build a tree structure data block index inside of the HFile

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.92.0, 0.94.0
    • Component/s: None
    • Labels:
      None

      Description

      The motivation is the data block index is too large to hold into memory. In our cases, each region server hold almost 6G block index in the memory. So we propose to build a tree structure data block index and only hold the root level in the memory. All the intermediate level will be cached in an LRU cache. In the leaf level, we will write the block index inline inside of the data section. In this way, there is no need to hold all the block index in the memory.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                liyin Liyin Tang
                Reporter:
                liyin Liyin Tang
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: