[RATIS-1708] RaftLog file channel closed before async flush ends

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.3.0
• Fix Version/s: 3.0.0, 2.4.1
• Target Version/s: 2.4.0
• Assignee: Song Ziyang
• Reporter: Song Ziyang
• Created: 14/Sep/22 06:00
• Updated: 25/Oct/22 10:17
• Resolved: 25/Oct/22 10:05

Description

When remove a peer from current raft group, we receive flooding error message

 

2022-09-13 16:31:07,877 [192.168.130.5_40010@group-000100000004-SegmentedRaftLogWorker-flush-thread1] ERROR o.a.r.s.r.s.BufferedWriteChannel:115 - Failed to flush channel
java.nio.channels.ClosedChannelException: null
        at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:110)
        at sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:379)
        at org.apache.ratis.server.raftlog.segmented.BufferedWriteChannel.fileChannelForce(BufferedWriteChannel.java:113)
        at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

 

After checking the code, we think that the raft log is closed before asyncFlush future returns. That is, close() should wait the result in (https://github.com/apache/ratis/blob/b7b87340b9ef5197fd751dd1a5dd1f9b68b8ffab/ratis-server/src/main/java/org/apache/ratis/server/raftlog/segmented/BufferedWriteChannel.java#L106) 

Issue Links

• links to: GitHub Pull Request #746