Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-11165 Scaling so cluster can host 1M regions and beyond (50M regions?)
  3. HBASE-14090

Redo FS layout; let go of tables/regions/stores directory hierarchy in DFS

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • Filesystem Integration
    • None

    Description

      Our layout as is won't work if 1M regions; e.g. HDFS will fall over if directories of hundreds of thousands of files. HBASE-13991 (Humongous Tables) would address this specific directory problem only by adding subdirs under table dir but there are other issues with our current layout:

      • Our table/regions/column family 'facade' has to be maintained in two locations – in master memory and in the hdfs directory layout – and the farce needs to be kept synced or worse, the model management is split between master memory and DFS layout. 'Syncing' in HDFS has us dropping constructs such as 'Reference' and 'HalfHFiles' on split, 'HFileLinks' when archiving, and so on. This 'tie' makes it hard to make changes.
      • While HDFS has atomic rename, useful for fencing and for having files added atomically, if the model were solely owned by hbase, there are hbase primitives we could make use of – changes in a row are atomic and coprocessors – to simplify table transactions and provide more consistent views of our model to clients; file 'moves' could be a memory operation only rather than an HDFS call; sharing files between tables/snapshots and when it is safe to remove them would be simplified if one owner only; and so on.

      This is an umbrella blue-sky issue to discuss what a new layout would look like and how we might get there. I'll follow up with some sketches of what new layout could look like that come of some chats a few of us have been having. We are also under the 'delusion' that move to a new layout could be done as part of a rolling upgrade and that the amount of work involved is not gargantuan.

      Attachments

        Issue Links

          is depended upon by

          New Feature - A new feature of the product, which has yet to be developed. HBASE-7218 Rename Snapshot

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          is related to

          Sub-task - The sub-task of the issue HBASE-14413 Procedure V2 - Snapshot V2

          • Major - Major loss of function.
          • Closed

          Umbrella - An overarching type made of sub-tasks HBASE-24749 Direct insert HFiles and Persist in-memory HFile tracking

          • Major - Major loss of function.
          • Closed
          relates to

          Sub-task - The sub-task of the issue HBASE-13991 Hierarchical Layout for Humongous Tables

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-14439 New/Improved Filesystem Abstractions

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Task - A task that needs to be done. HBASE-7806 Isolate the FileSystem calls

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Brainstorming - A place to record back and forth on notions not yet formed enough to make a 'New Feature' or 'Task' HBASE-13159 Consider RangeReferenceFiles with transformations

          • Major - Major loss of function.
          • Closed
          links to

          Web Link Discussion on new radically different approach to HBase FS directory layout REDO work

          Web Link FS Layout Redo Sketch

          Web Link Inmemory PoC

          Web Link Old FS Redo Plan by Matteo from early 2016

          Activity

            People

              Unassigned Unassigned
              stack Michael Stack
              Votes:
              0 Vote for this issue
              Watchers:
              32 Start watching this issue

              Dates

                Created:
                Updated: