[CASSANDRA-3533] TimeoutException when there is a firewall issue.

• Type: Improvement
• Status: Resolved
• Priority: Low
• Resolution: Fixed
• Fix Version/s: 2.0 beta 1
• Assignee: Vijay
• Reporter: Vijay

Authors

Vijay

Reviewers

Jonathan Ellis

• Created: 26/Nov/11 01:52
• Updated: 16/Apr/19 09:32
• Resolved: 10/Apr/13 17:23

Description

When one node in the cluster is not able to talk to the other DC/RAC due to firewall or network related issue (StorageProxy calls fail), and the nodes are NOT marked down because at least one node in the cluster can talk to the other DC/RAC, we get timeoutException instead of throwing a unavailableException.

The problem with this:
1) It is hard to monitor/identify these errors.
2) It is hard to diffrentiate from the client if the node being bad vs a bad query.
3) when this issue happens we have to wait for at-least the RPC timeout time to know that the query wont succeed.

Possible Solution: when marking a node down we might want to check if the node is actually alive by trying to communicate to it? So we can be sure that the node is actually alive.