[HBASE-14070] Hybrid Logical Clocks for HBase

• Type: New Feature
• Status: Reopened
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Enis Soztutar
• Created: 14/Jul/15 02:22
• Updated: 11/Jun/22 18:33

Description

HBase and Phoenix uses systems physical clock (PT) to give timestamps to events (read and writes). This works mostly when the system clock is strictly monotonically increasing and there is no cross-dependency between servers clocks. However we know that leap seconds, general clock skew and clock drift are in fact real.

This jira proposes using Hybrid Logical Clocks (HLC) as an implementation of hybrid physical clock + a logical clock. HLC is best of both worlds where it keeps causality relationship similar to logical clocks, but still is compatible with NTP based physical system clock. HLC can be represented in 64bits.

A design document is attached and also can be found here:
https://docs.google.com/document/d/1LL2GAodiYi0waBz5ODGL4LDT4e_bXy8P9h6kWC05Bhw/edit#

Issue Links

• incorporates: HBASE-16148 Hybrid Logical Clocks(placeholder for running tests)
• is related to: PHOENIX-2124 Hybrid Logical Clocks for Phoenix
• is related to: HBASE-9879 Can't undelete a KeyValue
• is related to: HBASE-14054 Acknowledged writes may get lost if regionserver clock is set backwards
• is related to: HBASE-9905 Enable using seqId as timestamp
• relates to: HBASE-6833 [WINDOWS] Java Milisecond precision on Windows
• relates to: HBASE-8927 Use nano time instead of mili time everywhere
• relates to: HBASE-10247 Client promises about timestamps
• links to: Current Status of HLC
• links to: review board
• links to: Review Board (master)

Sub-Tasks

• Add Timestamp class to the hbase-common: Open
• Hybrid Logical Clocks(placeholder for running tests): Open
• Initial patch (Core HLC) for public branch: Resolved
• Undoing using the master's timestamp for meta updates: Resolved
• Enable HLC for just the meta table: Resolved
• Fix TestTimestampType failing on isLikelyOfType tests: Resolved
• Update clock on region open and close: Resolved
• Design improvements to Clock.java: Resolved
• Add clock type to proto message for HLC region open/close: Resolved
• Update clock on replaying recovered edits: Resolved
• Remove usages of masterSystemTime: Resolved
• [HLC] Fix timing out tests in HBASE-14070.HLC branch: Resolved
• [HLC] Finishing cleanups: Resolved
• [HLC] Performance microbenchmarks: Open
• [HLC] Fix TestCellACLWithMultipleVersions timing out: Open
• Prevent clock from getting stuck after update(): Open
• [HLC] Move clocks to a separate o.a.h.h.clocks package: Open
• Improve HLC implementation performance: Open
• [HLC] Follow up work of updating clocks on assign/unassign (duplicate): Resolved
• Design improvements to Clock.java (duplicate): Resolved