Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-17314

Limit total buffered size for all replication sources

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.5.0, 2.0.0
    • Component/s: Replication
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Add a conf "replication.total.buffer.quota" to limit total size of buffered entries in all replication peers. It will prevent server getting OOM if there are many peers. Default value is 256MB.

      Description

      If we have many peers or some servers have many recovered queues, we will hold many entries in memory which will increase the pressure of GC, even maybe OOM because we will read entries for 64MB to buffer in default for one source.

        Attachments

        1. HBASE-17314.branch-1.v02.patch
          17 kB
          Phil Yang
        2. HBASE-17314.v06.patch
          17 kB
          Phil Yang
        3. HBASE-17314.v05.patch
          18 kB
          Phil Yang
        4. HBASE-17314.branch-1.v01.patch
          18 kB
          Phil Yang
        5. HBASE-17314.v04.patch
          17 kB
          Phil Yang
        6. HBASE-17314.v03.patch
          16 kB
          Phil Yang
        7. HBASE-17314.v02.patch
          16 kB
          Phil Yang
        8. HBASE-17314.v01.patch
          13 kB
          Phil Yang

          Activity

            People

            • Assignee:
              yangzhe1991 Phil Yang
              Reporter:
              yangzhe1991 Phil Yang
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: