[CASSANDRA-8448] "Comparison method violates its general contract" in AbstractEndpointSnitch

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 2.0.13, 2.1.3
• Severity: Normal
• Assignee: Brandon Williams
• Reporter: J.B. Langston

Authors

Brandon Williams

Reviewers

Benedict Elliott Smith

• Created: 09/Dec/14 21:31
• Updated: 16/Apr/19 09:31
• Resolved: 06/Feb/15 19:28

Description

Seen in both 1.2 and 2.0. The error is occurring here: https://github.com/apache/cassandra/blob/cassandra-2.0/src/java/org/apache/cassandra/locator/AbstractEndpointSnitch.java#L49

ERROR [Thrift:9] 2014-12-04 20:12:28,732 CustomTThreadPoolServer.java (line 219) Error occurred during processing of message.
com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: Comparison method violates its general contract!
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2199)
	at com.google.common.cache.LocalCache.get(LocalCache.java:3932)
	at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3936)
	at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4806)
	at org.apache.cassandra.service.ClientState.authorize(ClientState.java:352)
	at org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:224)
	at org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:218)
	at org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:202)
	at org.apache.cassandra.thrift.CassandraServer.createMutationList(CassandraServer.java:822)
	at org.apache.cassandra.thrift.CassandraServer.batch_mutate(CassandraServer.java:954)
	at com.datastax.bdp.server.DseServer.batch_mutate(DseServer.java:576)
	at org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3922)
	at org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3906)
	at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
	at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
	at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:201)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Comparison method violates its general contract!
	at java.util.TimSort.mergeHi(TimSort.java:868)
	at java.util.TimSort.mergeAt(TimSort.java:485)
	at java.util.TimSort.mergeCollapse(TimSort.java:410)
	at java.util.TimSort.sort(TimSort.java:214)
	at java.util.TimSort.sort(TimSort.java:173)
	at java.util.Arrays.sort(Arrays.java:659)
	at java.util.Collections.sort(Collections.java:217)
	at org.apache.cassandra.locator.AbstractEndpointSnitch.sortByProximity(AbstractEndpointSnitch.java:49)
	at org.apache.cassandra.locator.DynamicEndpointSnitch.sortByProximityWithScore(DynamicEndpointSnitch.java:157)
	at org.apache.cassandra.locator.DynamicEndpointSnitch.sortByProximityWithBadness(DynamicEndpointSnitch.java:186)
	at org.apache.cassandra.locator.DynamicEndpointSnitch.sortByProximity(DynamicEndpointSnitch.java:151)
	at org.apache.cassandra.service.StorageProxy.getLiveSortedEndpoints(StorageProxy.java:1408)
	at org.apache.cassandra.service.StorageProxy.getLiveSortedEndpoints(StorageProxy.java:1402)
	at org.apache.cassandra.service.AbstractReadExecutor.getReadExecutor(AbstractReadExecutor.java:148)
	at org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1223)
	at org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1165)
	at org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:255)
	at org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:225)
	at org.apache.cassandra.auth.Auth.selectUser(Auth.java:243)
	at org.apache.cassandra.auth.Auth.isSuperuser(Auth.java:84)
	at org.apache.cassandra.auth.AuthenticatedUser.isSuper(AuthenticatedUser.java:50)
	at org.apache.cassandra.auth.CassandraAuthorizer.authorize(CassandraAuthorizer.java:69)
	at org.apache.cassandra.service.ClientState$1.load(ClientState.java:338)
	at org.apache.cassandra.service.ClientState$1.load(ClientState.java:335)
	at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3522)
	at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2315)
	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2278)
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2193)
	... 18 more

Workaround: Setting -Djava.util.Arrays.useLegacyMergeSort=true causes the error to go away.

Issue Links

• is related to: CASSANDRA-9519 CASSANDRA-8448 Doesn't seem to be fixed