[HBASE-19673] Backport " Periodically ensure records are not buffered too long by BufferedMutator" to branch-1

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.5.0
• Component/s: Client
• Hadoop Flags: Reviewed

Release Note

The BufferedMutator now supports two settings that are used to ensure records do not stay too long in the buffer of a BufferedMutator. For periodically flushing the BufferedMutator there is now a "Timeout": "How old may the oldest record in the buffer be before we force a flush" and a "TimerTick": How often do we check if the timeout has been exceeded. Using these settings you can make the BufferedMutator automatically flush the write buffer if after the specified number of milliseconds no flush has occurred.

This is mainly useful in streaming scenarios (i.e. writing data into HBase using Apache Flink/Beam/Storm) where it is common (especially in a test/development situation) to see small unpredictable bursts of data that need to be written into HBase. When using the BufferedMutator till now the effect was that records would remain in the write buffer until the buffer was full or an explicit flush was triggered. In practice this would mean that the 'last few records' of a burst would remain in the write buffer until the next burst arrives filling the buffer to capacity and thus triggering a flush.

• Assignee: Niels Basjes
• Reporter: Niels Basjes
• Created: 30/Dec/17 09:29
• Updated: 24/Jun/22 19:06
• Resolved: 06/Jan/18 08:18

Description

In HBASE-19486 the feature was built to periodically flush the BufferedMutator.
Because backwards compatibility is important in the 1.x branch some refactoring is needed to make this work.

As requested by chia7712 this separate issue was created

Issue Links

• is related to: HBASE-19486 Periodically ensure records are not buffered too long by BufferedMutator