[CASSANDRA-6512] Thrift timeouts if node becomes unavailable

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Not A Problem
• Severity: Normal
• Since Version: 1.2.12
• Assignee: Unassigned
• Reporter: Adrian Adshead
• Created: 19/Dec/13 19:57
• Updated: 16/Apr/19 09:31
• Resolved: 19/Dec/13 20:04

Description

Create a cluster and add a keyspace and column family with data in it.
In cassandra-cli:
use KEYSPACE;
list COLUMNFAMILY;

it returns the data.

On another node:
Run 'service network stop && sleep 5 && service network start'

Back on the original node:
In cassandra-cli:
list COLUMNFAMILY;

Result is a timeout on thrift.

After the network is back up on the other node, then everything
returns to normal.

This operated correctly on 1.2.11, but fails in 1.2.12

Here is an example run:-

[default@adsheaddummy] list Users;
Using default limit of 100
Using default cell limit of 100
-------------------
RowKey: 635b67d0-68de-11e3-8b42-0a73bd7df27a
=> (name=email, value=[e-mail], timestamp=1387479011178000)
=> (name=name, value=Admin User, timestamp=1387479011150000)

1 Row Returned.
Elapsed time: 272 msec(s).
[default@adsheaddummy] list Users;
Using default limit of 100
Using default cell limit of 100
-------------------
RowKey: 635b67d0-68de-11e3-8b42-0a73bd7df27a
=> (name=email, value=[e-mail], timestamp=1387479011178000)
=> (name=name, value=Admin User, timestamp=1387479011150000)

1 Row Returned.
Elapsed time: 515 msec(s).
[default@adsheaddummy] list Users; <THIS IS WHEN THE NETWORK IS DOWN>
Using default limit of 100
Using default cell limit of 100
null
TimedOutException()
at org.apache.cassandra.thrift.Cassandra$get_range_slices_result.read(Cassandra.java:12932)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
at org.apache.cassandra.thrift.Cassandra$Client.recv_get_range_slices(Cassandra.java:734)
at org.apache.cassandra.thrift.Cassandra$Client.get_range_slices(Cassandra.java:718)
at org.apache.cassandra.cli.CliClient.executeList(CliClient.java:1495)
at org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:279)
at org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:213)
at org.apache.cassandra.cli.CliMain.main(CliMain.java:343)
[default@adsheaddummy] list Users; <NETWORK BACK UP>
Using default limit of 100
Using default cell limit of 100
-------------------
RowKey: 635b67d0-68de-11e3-8b42-0a73bd7df27a
=> (name=email, value=[e-mail], timestamp=1387479011178000)
=> (name=name, value=Admin User, timestamp=1387479011150000)

1 Row Returned.
Elapsed time: 341 msec(s).