Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-21641

RegionServer Group based WAL creation & replication

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: wal
    • Labels:
      None

      Description

      I'm looking at extending HBASE-6721 to apply it to WALs such that WALs are created & replicated within an RSGroup. This extends multi-tenancy to WALs also, and not just cover Hbase data. I was working out of 1.2.x code. The feedback on mailing list was to continue the discussion on Jira and target 2.x
       
      The approach I'm using is

      • Strategy interface for WAL placement on the filesystem. Default to delegate it to respective filesystem (which is the old behavior). FavoredNode strategy computes the favoured nodes from the RSGroup memberships.
      • FavoredNode strategy requires instance of hbase.Server, to get the current server name and an instance of Zookeeper watch to listen for changes to RSGroup memberships
      • The strategy is initialised in HRegionServer init and set in a static field in DefaultWALProvider
        - DefaultWALProvider.Writer takes the strategy in its init, and invokes it before output stream creation and passes the favored nodes information to DistributedFileSystem.create()
         
        Few questions
      • Any glaring miss in the approach?
      • I have hesitation in setting the strategy in static field in DefaultWALProvider. I would have preferred it to be passed in "init" itself, but that change seems to be too expansive.
      • Also, this introduces the dependency of server/zookeeper instance inside the WAL code path, which seems to be not there till now. Is that an explicit choice to keep them separate?

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              zodvik Nikhil Bafna
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated: