Details
-
Bug
-
Status: Resolved
-
Normal
-
Resolution: Fixed
-
None
-
None
-
Normal
Description
This is most evident with intermittent failures of the short_read dtests. I'll focus on short_read_reversed_test for explanation, since that's what I used to bisect. This test inserts some columns into a row, then deletes a subset, but it performs each delete on a different node, with another node down (hints are disabled.) Finally it reads the row back at QUORUM and checks that it doesn't see any deleted columns, however with speculative retry on this often fails. I bisected this to the change that made 99th percentile SR the default reliably by looping the test enough times at each iteration to be sure it was passing or failing.