On Linux platforms, the insertion of a leap second breaks the monotonicity of timestamps. This can make values appear to have been inserted into Cassandra in a different order than they were. I want to know what behavior is expected and desirable for inserts over this discontinuity.
From a timestamp perspective, an inserted leap second looks like a repeat of the previous second:
Note that 23:59:59 repeats itself, and that the timestamps increase during the first time through, then step back down to the beginning of the second and increase again.
As a result, the timestamps on values inserted during these seconds will be out of order. I set up a 4-node cluster running under Ubuntu 12.04.3 and synced them to shortly before the leap second would be inserted. During the insertion of the leap second, I ran a test with logic something like:
EDIT: This behavior occurs with server-generated timestamps; in this particular test, I set use_client_timestamp to False.
Under normal circumstances, the values and writetimes would increase together, but when inserted over the leap second, they don't. These value, writetime pairs are sorted by writetime:
The values were inserted in increasing order, but their writetimes are in a different order because of the repeated second. During the first instance of 23:59:59, the values 579, 580, and 581 were inserted at the beginning, middle, and end of the second. During the leap second, which is also 23:59:59, 582, 583, and 584 were inserted, also at the beginning, middle, and end of the second. However, since the two seconds are the same second, they appear interleaved with respect to timestamps, as shown above.
So, should I consider this behavior correct? If not, how should Cassandra correctly handle the discontinuity introduced by the insertion of a leap second?