[HBASE-14460] [Perf Regression] Merge of MVCC and SequenceId (HBASE-8763) slowed Increments, CheckAndPuts, batch operations

• Type: Umbrella
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.2.0, 1.3.0, 1.1.4, 1.0.4, 2.0.0
• Component/s: Performance

Release Note

This release note tries to tell the general story. Dive into sub-tasks for more specific release noting.

Increments, appends, checkAnd* have been slow since hbase-.1.0.0. The unification of mvcc and sequence id done by HBASE-8763 was responsible.

A ‘fast-path’ workaround was added by HBASE-15031 “Fix merge of MVCC and SequenceID performance regression in branch-1.0 for Increments”. It became available in 1.0.3 and 1.1.3. To enable the fast path, set "hbase.increment.fast.but.narrow.consistency" and then rolling restart. The workaround was for increments only (appends, checkAndPut, etc., were not addressed. See HBASE-15031 release note for more detail).

Subsequently, the regression was properly identified and fixed in HBASE-15213 and the fix applied to branch-1.0 and branch-1.1. As it happens, hbase-1.2.0 does not suffer from the performance regression (though the thought was that it did -- and so it got the fast-path patch too via HBASE-15092) nor does the master branch. HBASE-15213 identified that HBASE-12751 (as a side effect) had cured the regression.

hbase-1.0.4 (if it is ever released -- 1.0 has been end-of-lifed) and hbase-1.1.4 will have the HBASE-15213 fix. If you are suffering from the increment regression and you are on 1.0.3 or 1.1.3, you can enable the work around to get back your increment performance but you should upgrade.

• Assignee: Michael Stack
• Reporter: Michael Stack
• Created: 22/Sep/15 05:08
• Updated: 01/Jul/22 21:35
• Resolved: 08/Feb/16 18:47

Description

As reported by 鈴木俊裕 up on the mailing list – see "Performance degradation between CDH5.3.1(HBase0.98.6) and CDH5.4.5(HBase1.0.0)" – our unification of sequenceid and MVCC slows Increments (and other ops) as the mvcc needs to 'catch up' to our current point before we can read the last Increment value that we need to update.

We can say that our Increment is just done wrong, we should just be writing Increments and summing on read, but checkAndPut as well as batching operations have the same issue. Fix.

Issue Links

• is broken by: HBASE-8763 Combine MVCC and SeqId
• relates to: HBASE-15213 Fix increment performance regression caused by HBASE-8763 on branch-1.0

Sub-Tasks

• Fix merge of MVCC and SequenceID performance regression in branch-1.0 for Increments: Closed
• Perf test doing all mutation steps under row lock: Resolved
• Fix merge of MVCC and SequenceID performance regression: Resolved
• Forward-port to 1.2+ HBASE-15031 "Fix merge of MVCC and SequenceID performance regression in branch-1.0 for Increments": Resolved
• isReturnResult=false on fast path in branch-1.1 and branch-1.0 is not respected: Closed
• Apply checkFamilies addendum on increment to 1.1 and 1.0: Resolved
• Add *PerformanceTest for Append, CheckAnd*: Resolved
• Change order in which we do write pipeline operations; do all under row locks!: Resolved
• Fix merge of MVCC and SequenceID performance regression in branch-1.0 for checkAnd* and Append: Resolved
• Fix increment performance regression caused by HBASE-8763 on branch-1.0: Resolved
• Undo "hbase.increment.fast.but.narrow.consistency" option; it is not necessary since HBASE-15213: Resolved