[CASSANDRA-5133] Nodes can't rejoin after stopping, when using GossipingPropertyFileSnitch

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 1.2.1

Environment

3 ec2 instances (CentOS 6.3; java 1.7.0_05; Cassandra 1.2)

• Severity: Normal
• Assignee: Brandon Williams
• Reporter: Matt Jurik

Authors

Brandon Williams

Reviewers

Vijay

• Created: 08/Jan/13 23:10
• Updated: 16/Apr/19 09:32
• Resolved: 16/Jan/13 18:29

Description

I can establish a 1.2 ring with GossipingPropertyFileSnitch, but after killing a node and restarting it, the node cannot rejoin.

[Node 1] ./bin/cassandra -f
[Node 2] ./bin/cassandra -f
[Node 3] ./bin/cassandra -f

[Node 1] ./bin/nodetool ring
... ok ...

[Node 1] ^C
... node shutdown ...

[Node 1] ./bin/cassandra -f
... Exception! ...

ERROR 05:45:39,305 Exception encountered during startup
java.lang.RuntimeException: Could not retrieve DC for /10.114.18.51 from gossip and PFS compatibility is disabled
at org.apache.cassandra.locator.GossipingPropertyFileSnitch.getDatacenter(GossipingPropertyFileSnitch.java:109)
at org.apache.cassandra.locator.DynamicEndpointSnitch.getDatacenter(DynamicEndpointSnitch.java:127)
at org.apache.cassandra.locator.TokenMetadata$Topology.addEndpoint(TokenMetadata.java:1040)
at org.apache.cassandra.locator.TokenMetadata.updateNormalTokens(TokenMetadata.java:185)
at org.apache.cassandra.locator.TokenMetadata.updateNormalTokens(TokenMetadata.java:157)
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:441)
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:397)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:309)
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:397)
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:440)
java.lang.RuntimeException: Could not retrieve DC for /10.114.18.51 from gossip and PFS compatibility is disabled
at org.apache.cassandra.locator.GossipingPropertyFileSnitch.getDatacenter(GossipingPropertyFileSnitch.java:109)
at org.apache.cassandra.locator.DynamicEndpointSnitch.getDatacenter(DynamicEndpointSnitch.java:127)
at org.apache.cassandra.locator.TokenMetadata$Topology.addEndpoint(TokenMetadata.java:1040)
at org.apache.cassandra.locator.TokenMetadata.updateNormalTokens(TokenMetadata.java:185)
at org.apache.cassandra.locator.TokenMetadata.updateNormalTokens(TokenMetadata.java:157)
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:441)
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:397)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:309)
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:397)
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:440)

Full environment + exceptions + stacktraces: https://gist.github.com/1e74ff02c2d4f622ce8f