HBase
  1. HBase
  2. HBASE-3144

Freeze and thaw for regions and/or regionservers

    Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: regionserver
    • Labels:
      None

      Description

      When frozen, a region would not flush new files or delete old files.

      Context:
      Was talking to Dhruba about the possibility of a fast intra-cluster copy, and he says it is totally do-able. The idea is this: if there is a HBase table that is very large (multiple terabytes) then distcp takes a long time. The major bottleneck is the cross rack network transfer time. We can easily avoid this by going to each of the block replicas and replicating them on the same machine from one disk to another. For our infrastructure setup, this could make hours of copy time seconds.

      If we are able to implement the above, then copying an entire HBase table will only take seconds. Now we just have to ensure that the region files do not "change" during this period - so we need to disable compactions and flushes. Hence the need for the freeze and thaw of regions in a region server.

      The way this would work is to flush a region, freeze it, snapshot it, then thaw it and let normal operations proceed. The snapshot would proceed at disk read/write speed so it should take only seconds per region.

        Activity

        Hide
        Andrew Purtell added a comment -

        This covered by HBASE-50?

        Show
        Andrew Purtell added a comment - This covered by HBASE-50 ?
        Karthik Ranganathan created issue -

          People

          • Assignee:
            Unassigned
            Reporter:
            Karthik Ranganathan
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:

              Development