[PHOENIX-4089] Prevent index from getting out of sync with data table under high concurrency

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.12.0
• Labels: secondary_index
• Assignee: James R. Taylor
• Reporter: James R. Taylor
• Created: 16/Aug/17 15:44
• Updated: 17/Oct/18 17:15
• Resolved: 18/Aug/17 00:27

Description

Under high concurrency, we're still seeing the index get out of sync with the data table. It seems that the particular case is when the same Put occurs with the same time stamp from different clients, based on the locking we do, Phoenix thinks a different Put was the last one than HBase does, leading to inconsistencies.

The solution is to timestamp the cells on the server-side after the lock has been taken. The new concurrent unit test passes 50x with this in place, while it otherwise fails 1/10 of the time (or more on HBase 1.3).

Issue Links

• breaks: PHOENIX-4124 Two new tests without Apache licenses
• relates to: PHOENIX-4980 Mismatch in row counts between data and index tables while multiple clients try to upsert data
• relates to: PHOENIX-4849 Phoenix may incorrectly replace TableResultIterators after HBase region splits