Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-14379 Replication V2
  3. HBASE-15995

Separate replication WAL reading from shipping

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.0.0
    • 1.4.0, 2.0.0
    • Replication
    • None
    • Reviewed

    Description

      Currently ReplicationSource reads edits from the WAL and ships them in the same thread.

      By breaking out the reading from the shipping, we can introduce greater parallelism and lay the foundation for further refactoring to a pipelined, streaming model.

      Attachments

        1. HBASE-15995.master.v1.patch
          96 kB
          Vincent Poon
        2. replicationV1_100ms_delay.png
          52 kB
          Vincent Poon
        3. replicationV2_100ms_delay.png
          64 kB
          Vincent Poon
        4. HBASE-15995.master.v2.patch
          116 kB
          Vincent Poon
        5. HBASE-15995.master.v3.patch
          116 kB
          Vincent Poon
        6. HBASE-15995.master.v4.patch
          116 kB
          Vincent Poon
        7. HBASE-15995.master.v6.patch
          116 kB
          Vincent Poon
        8. HBASE-15995.master.v7.patch
          116 kB
          Vincent Poon
        9. HBASE-15995.branch-1.v7.patch
          116 kB
          Vincent Poon

        Issue Links

        is related to

        Improvement - An improvement or enhancement to an existing feature or task. HBASE-17432 Use less threads to read WAL in replication

        • Major - Major loss of function.
        • Open

        Task - A task that needs to be done. HBASE-24779 Improve insight into replication WAL readers hung on checkQuota

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved
        links to

        Web Link review board

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            vincentpoon Vincent Poon
            vincentpoon Vincent Poon
            Votes:
            0 Vote for this issue
            Watchers:
            14 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment