Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-12259

Bring quorum based write ahead log into HBase

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Won't Fix
    • 2.0.0
    • None
    • wal
    • None

    Description

      HydraBase ( https://code.facebook.com/posts/321111638043166/hydrabase-the-evolution-of-hbase-facebook/ ) Facebook's implementation of HBase with Raft for consensus will be going open source shortly. We should pull in the parts of that fb-0.89 based implementation, and offer it as a feature in whatever next major release is next up. Right now the Hydrabase code base isn't ready to be released into the wild; it should be ready soon ( for some definition of soon).

      Since Hydrabase is based upon 0.89 most of the code is not directly applicable. So lots of work will probably need to be done in a feature branch before a merge vote.

      Is this something that's wanted?

      Is there anything clean up that needs to be done before the log implementation is able to be replaced like this?

      What's our story with upgrading to this? Are we ok with requiring down time ?

      Attachments

        1. RaftProtocolImplementationDesignDoc.pdf
          980 kB
          Rishit Shroff
        2. Architecture for HydraBase (5).pdf
          138 kB
          Rishit Shroff
        1.
        HydraBase Consensus Protocol Sub-task Closed Unassigned   Actions
        2.
        Integrate HydraBase RMap in RegionManager Sub-task Closed Unassigned   Actions
        3.
        Integrate the RAFT Protocol into WAL Sub-task Closed Rishit Shroff   Actions
        4.
        Update the read/write path of HBase w.r.t the RAFT Protocol Sub-task Closed Unassigned   Actions
        5.
        Update the HBase client to be Quorum aware Sub-task Closed Unassigned   Actions
        6.
        Integrate HRegion Role policy (ACTIVE|ACTIVE-WITNESS|SHADOW-WITNESS) Sub-task Closed Unassigned   Actions
        7.
        Integrate replication into HydraBase Sub-task Closed Unassigned   Actions
        8.
        Integrate Quorum Membership change into HBase Sub-task Closed Unassigned   Actions
        9.
        Make flushes and compactions work with quorum based WAL Sub-task Closed Unassigned   Actions
        10.
        Export RAFT protocol metrics Sub-task Closed Unassigned   Actions
        11.
        Modify HBCK to work with Quorum based WAL Sub-task Closed Unassigned   Actions
        12.
        Describe the upgrade procedure from HBase to HydraBase Sub-task Closed Unassigned   Actions
        13.
        Make table admin operations work with HydraBase Sub-task Closed Unassigned   Actions
        14.
        Remove the dependency on HRegionInfo from 0.89-fb Sub-task Closed Gaurav Menghani   Actions
        15.
        Move from HServerAddress (0.89-fb) to HostAndPort in hbase-consensus module Sub-task Closed Unassigned   Actions
        16.
        Remove AbstractWAL from hbase-consensus Sub-task Closed Unassigned   Actions
        17.
        Add license headers to the java files hbase-consensus module using the apache rat tool Sub-task Closed Rishit Shroff

        0%

        Original Estimate - 24h
        Remaining Estimate - 24h
        Actions
        18.
        Deprecate 0.89-fb specific Data Structures like KeyValue, WALEdit etc Sub-task Closed Unassigned   Actions

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            eclark Elliott Neil Clark
            Votes:
            5 Vote for this issue
            Watchers:
            76 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - 24h
                24h
                Remaining:
                Remaining Estimate - 24h
                24h
                Logged:
                Time Spent - Not Specified
                Not Specified

                Slack

                  Issue deployment