Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-14379 Replication V2
  3. HBASE-15995

Separate replication WAL reading from shipping

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.0.0
    • 1.4.0, 2.0.0
    • Replication
    • None
    • Reviewed

    Description

      Currently ReplicationSource reads edits from the WAL and ships them in the same thread.

      By breaking out the reading from the shipping, we can introduce greater parallelism and lay the foundation for further refactoring to a pipelined, streaming model.

      Attachments

        1. HBASE-15995.master.v1.patch
          96 kB
          Vincent Poon
        2. replicationV1_100ms_delay.png
          52 kB
          Vincent Poon
        3. replicationV2_100ms_delay.png
          64 kB
          Vincent Poon
        4. HBASE-15995.master.v2.patch
          116 kB
          Vincent Poon
        5. HBASE-15995.master.v3.patch
          116 kB
          Vincent Poon
        6. HBASE-15995.master.v4.patch
          116 kB
          Vincent Poon
        7. HBASE-15995.master.v6.patch
          116 kB
          Vincent Poon
        8. HBASE-15995.master.v7.patch
          116 kB
          Vincent Poon
        9. HBASE-15995.branch-1.v7.patch
          116 kB
          Vincent Poon

        Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-17432 Use less threads to read WAL in replication

          • Major - Major loss of function.
          • Open

          Task - A task that needs to be done. HBASE-24779 Improve insight into replication WAL readers hung on checkQuota

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          links to

          Web Link review board

          Activity

            People

              vincentpoon Vincent Poon
              vincentpoon Vincent Poon
              Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: