Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
0.20.0
-
None
-
None
-
Reviewed
Description
ICV demonstrates a race condition under high load. The result is a duplicate KeyValue with the same timestamp, at first in the memcache, and in hfile, then both in hfile. The get/scan code doesnt know which one to read, and picks one arbitrarily. One of the keyvalues is correct, one is incorrect.
What happens at a deeper level:
- we start an ICV
- a snapshot happens and moves the memstore to the snapshot
- the ICV code puts a key-value into memstore that has the same timestamp as a keyvalue in the snapshot.
This is a deep race condition and several attempts to fix it failed in production here at SU. This issue is about a more permanent fix.