[CASSANDRA-11302] Invalid time unit conversion causing write timeouts

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 2.1.14, 2.2.6, 3.0.5, 3.5
• Component/s: Legacy/Core
• Severity: Normal
• Since Version: 2.1.5
• Assignee: Sylvain Lebresne
• Reporter: Mike Heffner

Authors

Sylvain Lebresne

Reviewers

Ariel Weisberg

• Created: 04/Mar/16 04:42
• Updated: 16/Apr/19 09:30
• Resolved: 08/Mar/16 16:26

Description

We've been debugging a write timeout that we saw after upgrading from the 2.0.x release line, with our particular workload. Details of that process can be found in this thread:

https://www.mail-archive.com/[e-mail]/msg46064.html

After bisecting various patch release versions, and then commits, on the 2.1.x release line we've identified version 2.1.5 and this commit as the point where the timeouts first start appearing:

https://github.com/apache/cassandra/commit/828496492c51d7437b690999205ecc941f41a0a9

After examining the commit we believe this line was a typo:

https://github.com/apache/cassandra/commit/828496492c51d7437b690999205ecc941f41a0a9#diff-c7ef124561c4cde1c906f28ad3883a88L467

as it doesn't properly convert the timeout value from milliseconds to nanoseconds.

After testing with the attached patch applied, we do not see timeouts on version 2.1.5 nor against 2.2.5 when we bring the patch forward. While we've tested our workload against this and we are fairly confident in the patch, we are not experts with the code base so we would prefer additional review.

Issue Links

• is broken by: CASSANDRA-8692 Coalesce intra-cluster network messages