[HBASE-7325] Replication reacts slowly on a lightly-loaded cluster

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 0.98.0, 0.95.2
• Component/s: Replication
• Hadoop Flags: Reviewed

Release Note

ReplicationSource won't backoff sleeping time if it can correctly read from the WAL even though it's not replicating anything. In that kind of situation it will put more load on the NN. replication.source.sleepforretries can still be tweaked to set the minimal sleep time.

• Assignee: Gabriel Reid
• Reporter: Gabriel Reid
• Created: 11/Dec/12 16:33
• Updated: 23/Sep/13 19:28
• Resolved: 07/Aug/13 16:32

Description

ReplicationSource uses a backing-off algorithm to sleep for an increasing duration when an error is encountered in the replication run loop. However, this backing-off is also performed when there is nothing found to replicate in the HLog.

Assuming default settings (1 second base retry sleep time, and maximum multiplier of 10), this means that replication takes up to 10 seconds to occur when there is a break of about 55 seconds without anything being written. As there is no error condition, and there is apparently no substantial load on the regionserver in this situation, it would probably make more sense to not back off in non-error situations.