[HBASE-11778] Scale timestamps by 1000

• Type: Brainstorming
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Tags: Phoenix
• Assignee: Unassigned
• Reporter: Lars Hofhansl
• Created: 19/Aug/14 20:12
• Updated: 17/Jun/22 17:03
• Resolved: 20/Aug/14 17:40

Description

The KV timestamps are used for various reasons:

1. ordering of KVs
2. resolving conflicts
3. enforce TTL

Currently we assume that the timestamps have a resolution of 1ms, and because of that we made the resolution at which we can determine time identical to the resolution at which we can store time.
I think it is time to disentangle the two... At least allow a higher resolution of time to be stored. That way we could have a centralized transaction oracle that produces ids that relate to wall clock time, and at the same time allow producing more than 1000/s.

The simplest way is to just store time in us (microseconds). I.e. we'd still collect time in ms by default and just multiply that with 1000 before we store it. With 8 bytes that still gives us a range of 292471 years.

We'd have grandfather in old data. Could write a metadata entry into each HFile declaring what the TS resolution is if it is different from ms.

Not sure, yet, how this would relate to using the TS for things like seqIds.

Let's do some brainstorming.