[RATIS-1634] SegmentedRaftLogReader not closed correctly

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.4.0, 3.0.0
• Component/s: server
• Assignee: Riguz Lee
• Reporter: Riguz Lee
• Created: 22/Jul/22 04:45
• Updated: 16/Aug/22 16:34
• Resolved: 22/Jul/22 17:24

Description

In SegmentedRaftLogInputStream#init, the SegmentedRaftLogReader instance is not correctly closed if the log is corrupted. The LogSegment#readSegmentFile function will try to delete empty segment, however it may fail on Windows because previously it's not closed. 

This error is caught by test case

ServerRestartTests#testRestartWithCorruptedLogHeader, which fails on Windows:

 

 

java.util.concurrent.CompletionException: java.lang.IllegalStateException: s0@group-FDE17229F06A: Failed to initRaftLog.
    at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:332)
    at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:347)
    at java.base/java.util.concurrent.CompletableFuture$BiRelay.tryFire(CompletableFuture.java:1498)
    at java.base/java.util.concurrent.CompletableFuture$CoCompletion.tryFire(CompletableFuture.java:1219)
    at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
    at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162)
    at org.apache.ratis.util.ConcurrentUtils.accept(ConcurrentUtils.java:176)
    at org.apache.ratis.util.ConcurrentUtils.lambda$parallelForEachAsync$3(ConcurrentUtils.java:165)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.IllegalStateException: s0@group-FDE17229F06A: Failed to initRaftLog.
    at org.apache.ratis.server.impl.ServerState.initRaftLog(ServerState.java:219)
    at org.apache.ratis.server.impl.ServerState.lambda$new$5(ServerState.java:160)
    at org.apache.ratis.util.MemoizedSupplier.get(MemoizedSupplier.java:62)
    at org.apache.ratis.server.impl.ServerState.initialize(ServerState.java:174)
    at org.apache.ratis.server.impl.RaftServerImpl.start(RaftServerImpl.java:310)
    at org.apache.ratis.util.ConcurrentUtils.accept(ConcurrentUtils.java:173)
    ... 4 more
Caused by: java.nio.file.FileSystemException: C:\Users\dr\Desktop_riguz\ratis\ratis-test\target\test\data\87bb11f4\MiniRaftClusterWithGrpcb13dc309\s0\0e749d95-adc0-4b0f-9268-fde17229f06a\current\log_inprogress_0: 另一个程序正在使用此文件，进程无法访问。
    at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:92)
    at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
    at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
    at java.base/sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:275)
    at java.base/sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:105)
    at java.base/java.nio.file.Files.delete(Files.java:1152)
    at org.apache.ratis.util.FileUtils.lambda$delete$8(FileUtils.java:156)
    at org.apache.ratis.util.LogUtils.runAndLog(LogUtils.java:38)
    at org.apache.ratis.util.FileUtils.delete(FileUtils.java:155)
    at org.apache.ratis.util.FileUtils.deleteFile(FileUtils.java:98)
    at org.apache.ratis.server.raftlog.segmented.LogSegment.loadSegment(LogSegment.java:186)
    at org.apache.ratis.server.raftlog.segmented.SegmentedRaftLogCache.loadSegment(SegmentedRaftLogCache.java:387)
    at org.apache.ratis.server.raftlog.segmented.SegmentedRaftLog.loadLogSegments(SegmentedRaftLog.java:241)
    at org.apache.ratis.server.raftlog.segmented.SegmentedRaftLog.openImpl(SegmentedRaftLog.java:214)
    at org.apache.ratis.server.raftlog.RaftLogBase.open(RaftLogBase.java:249)
    at org.apache.ratis.server.impl.ServerState.initRaftLog(ServerState.java:236)
    at org.apache.ratis.server.impl.ServerState.initRaftLog(ServerState.java:217)
    ... 9 more
Process finished with exit code -1

Issue Links

• links to: GitHub Pull Request #693