HBase
  1. HBase
  2. HBASE-9047

Tool to handle finishing replication when the cluster is offline

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.96.0
    • Fix Version/s: 0.98.0, 0.94.15, 0.96.2, 0.99.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      We're having a discussion on the mailing list about replicating the data on a cluster that was shut down in an offline fashion. The motivation could be that you don't want to bring HBase back up but still need that data on the slave.

      So I have this idea of a tool that would be running on the master cluster while it is down, although it could also run at any time. Basically it would be able to read the replication state of each master region server, finish replicating what's missing to all the slave, and then clear that state in zookeeper.

      The code that handles replication does most of that already, see ReplicationSourceManager and ReplicationSource. Basically when ReplicationSourceManager.init() is called, it will check all the queues in ZK and try to grab those that aren't attached to a region server. If the whole cluster is down, it will grab all of them.

      The beautiful thing here is that you could start that tool on all your machines and the load will be spread out, but that might not be a big concern if replication wasn't lagging since it would take a few seconds to finish replicating the missing data for each region server.

      I'm guessing when starting ReplicationSourceManager you'd give it a fake region server ID, and you'd tell it not to start its own source.

      FWIW the main difference in how replication is handled between Apache's HBase and Facebook's is that the latter is always done separately of HBase itself. This jira isn't about doing that.

      1. HBASE-9047-trunk-v0.patch
        23 kB
        Demai Ni
      2. HBASE-9047-0.94.9-v0.PATCH
        23 kB
        Demai Ni
      3. HBASE-9047-trunk-v1.patch
        22 kB
        Demai Ni
      4. HBASE-9047-trunk-v2.patch
        21 kB
        Demai Ni
      5. HBASE-9047-trunk-v3.patch
        21 kB
        Demai Ni
      6. HBASE-9047-trunk-v4.patch
        21 kB
        Demai Ni
      7. HBASE-9047-trunk-v4.patch
        21 kB
        stack
      8. HBASE-9047-trunk-v5.patch
        21 kB
        Demai Ni
      9. HBASE-9047-trunk-v6.patch
        21 kB
        Demai Ni
      10. HBASE-9047-trunk-v7.patch
        21 kB
        Demai Ni
      11. HBASE-9047-trunk-v7.patch
        21 kB
        Demai Ni
      12. HBASE-9047-0.94-v1.patch
        21 kB
        Demai Ni

        Issue Links

          Activity

          Jean-Daniel Cryans created issue -
          Demai Ni made changes -
          Field Original Value New Value
          Assignee Demai Ni [ nidmhbase ]
          stack made changes -
          Fix Version/s 0.95.2 [ 12320040 ]
          Fix Version/s 0.98.0 [ 12323143 ]
          Demai Ni made changes -
          Attachment hbase-9047-0.94.9-v0 [ 12598679 ]
          Demai Ni made changes -
          Attachment HBASE-9047-trunk-v0.patch [ 12602873 ]
          Demai Ni made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Demai Ni made changes -
          Attachment hbase-9047-0.94.9-v0 [ 12598679 ]
          Demai Ni made changes -
          Attachment HBASE-9047-0.94.9-v0.PATCH [ 12602902 ]
          Demai Ni made changes -
          Attachment HBASE-9047-trunk-v1.patch [ 12604424 ]
          Demai Ni made changes -
          Attachment HBASE-9047-trunk-v2.patch [ 12610459 ]
          Demai Ni made changes -
          Status In Progress [ 3 ] Patch Available [ 10002 ]
          Affects Version/s 0.96.0 [ 12324822 ]
          Fix Version/s 0.98.0 [ 12323143 ]
          Demai Ni made changes -
          Attachment HBASE-9047-trunk-v3.patch [ 12611148 ]
          Demai Ni made changes -
          Attachment HBASE-9047-trunk-v4.patch [ 12612459 ]
          stack made changes -
          Attachment HBASE-9047-trunk-v4.patch [ 12612659 ]
          Demai Ni made changes -
          Attachment HBASE-9047-trunk-v5.patch [ 12617306 ]
          Demai Ni made changes -
          Attachment HBASE-9047-trunk-v6.patch [ 12617970 ]
          Demai Ni made changes -
          Attachment HBASE-9047-trunk-v7.patch [ 12618067 ]
          Demai Ni made changes -
          Attachment HBASE-9047-trunk-v7.patch [ 12618067 ]
          Demai Ni made changes -
          Attachment HBASE-9047-trunk-v7.patch [ 12618077 ]
          Demai Ni made changes -
          Attachment HBASE-9047-trunk-v7.patch [ 12618460 ]
          Lars Hofhansl made changes -
          Fix Version/s 0.99.0 [ 12325675 ]
          Demai Ni made changes -
          Fix Version/s 0.96.1 [ 12324961 ]
          Fix Version/s 0.94.15 [ 12325559 ]
          Demai Ni made changes -
          Attachment HBASE-9047-0.94-v1.patch [ 12618538 ]
          stack made changes -
          Fix Version/s 0.96.1 [ 12324961 ]
          Lars Hofhansl made changes -
          Fix Version/s 0.96.2 [ 12325658 ]
          Lars Hofhansl made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Resolution Fixed [ 1 ]
          Lars Hofhansl made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Lars Hofhansl made changes -
          Link This issue relates to HBASE-10249 [ HBASE-10249 ]

            People

            • Assignee:
              Demai Ni
              Reporter:
              Jean-Daniel Cryans
            • Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development