Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-9753

LOCAL_QUORUM reads can block cross-DC if there is a digest mismatch



    • Bug
    • Status: Open
    • Normal
    • Resolution: Unresolved
    • None
    • Legacy/Coordination
    • None
    • Normal


      When there is a digest mismatch during the initial read, a data read request is sent to all replicas involved in the initial read. This can be more than the initial blockFor if read repair was done and if speculative retry kicked in. E.g. for RF 3 in two DCs, the number of reads could be 4: 2 for LOCAL_QUORUM, 1 for read repair and 1 for speculative read if one replica was slow. If there is then a digest mismatch, Cassandra will issue the data read to all 4 and set blockFor=4. Now the read query is blocked on cross-DC latency. The digest mismatch read blockFor should be capped at RF for the local DC when using CL.LOCAL_*.

      You can reproduce this behaviour by creating a keyspace with NetworkTopologyStrategy, RF 3 per DC, dc_local_read_repair=1.0 and ALWAYS for speculative read. If you force a digest mismatch (e.g. by deleting a replicas SSTables and restarting) you can see in tracing that it is blocking for 4 responses.


        Issue Links



              Unassigned Unassigned
              rlow Richard Low
              3 Vote for this issue
              22 Start watching this issue