[CASSANDRA-5830] Paxos loops endlessly due to faulty condition check

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Component/s: Feature/Lightweight Transactions
• Labels: LWT paxos
• Severity: Normal
• Since Version: 2.0 beta 1
• Assignee: Soumava Ghosh
• Reporter: Soumava Ghosh

Authors

Soumava Ghosh

Reviewers

Jonathan Ellis

• Created: 30/Jul/13 17:52
• Updated: 25/Oct/19 13:11
• Resolved: 05/Aug/13 21:44

Description

Following is the code segment (StorageProxy.java:361) which causes the issue:

Start is the start time of the paxos, is always less than the current system time, and therefore the negative difference is always less than the timeout.

StorageProxy.java

private static UUID beginAndRepairPaxos(long start, ByteBuffer key, CFMetaData metadata, List<InetAddress> liveEndpoints, int requiredParticipants, ConsistencyLevel consistencyForPaxos)
    throws WriteTimeoutException
    {
        long timeout = TimeUnit.MILLISECONDS.toNanos(DatabaseDescriptor.getCasContentionTimeout());

        PrepareCallback summary = null;
        while (start - System.nanoTime() < timeout)
        {
            long ballotMillis = summary == null
                              ? System.currentTimeMillis()
                              : Math.max(System.currentTimeMillis(), 1 + UUIDGen.unixTimestamp(summary.inProgressCommit.ballot));
            UUID ballot = UUIDGen.getTimeUUID(ballotMillis);

Here, the paxos gets stuck when PREPARE returns 'true' but with inProgressCommit. The code in StorageProxy.java:beginAndRepairPaxos() then tries to issue a PROPOSE and COMMIT for the inProgressCommit, and if it repeatedly receives 'false' as a PREPARE_RESPONSE it gets stuck in an endless loop until PREPARE_RESPONSE is true.