[HBASE-12609] Consider bandwidth throttling in ReplicationSource.readAllEntriesToReplicateOrNextFile

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Incomplete
• Affects Version/s: 0.98.8
• Component/s: Replication
• Assignee: Unassigned
• Reporter: Jianwei Cui
• Created: 02/Dec/14 04:49
• Updated: 17/Jun/22 18:26
• Resolved: 15/Jun/22 00:50

Description

The push size in ReplicationSource.shipEdits is controlled by 'replicationQueueSizeCapacity' and 'replicationQueueNbCapacity'. When the push size is much bigger than throttler's bandwidth(replication throttler is enabled), ReplicationSource will have a outgoing bandwidth peak followed by a period of sleep controlled by the throttler. How about considering bandwidth throttling in ReplicationSource.readAllEntriesToReplicateOrNextFile? such as the following code:

       // Stop if too many entries or too big
      long bandwidth = this.conf.getLong("replication.source.per.peer.node.bandwidth", 0);
      if ((bandwidth > 0 && currentSize > bandwidth) ||  // ==> also consider bandwidth throttling
          currentSize >= this.replicationQueueSizeCapacity ||
           entries.size() >= this.replicationQueueNbCapacity) {
         break;
       }

Then, ReplicationSource will have more steady outgoing bandwidth.