[CASSANDRA-14419] Resume compresed hints delivery broken

• Type: Bug
• Status: Resolved
• Priority: Low
• Resolution: Fixed
• Fix Version/s: 3.0.17
• Component/s: Consistency/Hints
• Severity: Low
• Since Version: 3.0.3
• Assignee: Tommy Stendahl
• Reporter: Tommy Stendahl

Authors

Tommy Stendahl

Reviewers

Aleksey Yeschenko

• Created: 26/Apr/18 09:08
• Updated: 16/Apr/19 09:29
• Resolved: 03/Jul/18 13:25

Description

We are using Cassandra 3.0.15 and are using compressed hints, but if hint delivery is interrupted resuming hint delivery is failing.

2018-04-04T13:27:48.948+0200 ERROR [HintsDispatcher:14] CassandraDaemon.java:207 Exception in thread Thread[HintsDispatcher:14,1,main]
java.lang.IllegalArgumentException: Unable to seek to position 1789149057 in /var/lib/cassandra/hints/9592c860-1054-4c60-b3b8-faa9adc6d769-1522838912649-1.hints (118259682 bytes) in read-only mode
        at org.apache.cassandra.io.util.RandomAccessReader.seek(RandomAccessReader.java:287) ~[apache-cassandra-clientutil-3.0.15.jar:3.0.15]
        at org.apache.cassandra.hints.HintsReader.seek(HintsReader.java:114) ~[apache-cassandra-3.0.15.jar:3.0.15]
        at org.apache.cassandra.hints.HintsDispatcher.seek(HintsDispatcher.java:83) ~[apache-cassandra-3.0.15.jar:3.0.15]
        at org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.deliver(HintsDispatchExecutor.java:263) ~[apache-cassandra-3.0.15.jar:3.0.15]
        at org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:248) ~[apache-cassandra-3.0.15.jar:3.0.15]
        at org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:226) ~[apache-cassandra-3.0.15.jar:3.0.15]
        at org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:205) ~[apache-cassandra-3.0.15.jar:3.0.15]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_152]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_152]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_152]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_152]
        at org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79) [apache-cassandra-3.0.15.jar:3.0.15]
        at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_152]

 I think the problem is similar to CASSANDRA-11960.