[RATIS-1887] Gap between segement log

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 3.0.0
• Component/s: server
• Assignee: GuoHao
• Reporter: GuoHao
• Created: 13/Sep/23 08:39
• Updated: 08/Nov/24 22:37
• Resolved: 18/Sep/23 05:58

Description

 

My version of ratis already includes this pr(https://issues.apache.org/jira/browse/RATIS-1763) and I am using a new raft server.

 

Description:
1. i am using ratis version 2.5.1
2. the application software is ozone 1.3.0 scm ha

 

scm error log:

Caused by: java.lang.IllegalStateException: Found a gap between logs: the last log segment log-784848_809981 ended at 809981 but the next log segment log-822560_856038 started at 822560
        at org.apache.ratis.util.Preconditions.assertTrue(Preconditions.java:72)
        at org.apache.ratis.server.raftlog.segmented.SegmentedRaftLogCache.validateAdding(SegmentedRaftLogCache.java:424)
        at org.apache.ratis.server.raftlog.segmented.SegmentedRaftLogCache.addSegment(SegmentedRaftLogCache.java:431)
        at org.apache.ratis.server.raftlog.segmented.SegmentedRaftLogCache.loadSegment(SegmentedRaftLogCache.java:384)
        at org.apache.ratis.server.raftlog.segmented.SegmentedRaftLog.loadLogSegments(SegmentedRaftLog.java:241)
        at org.apache.ratis.server.raftlog.segmented.SegmentedRaftLog.openImpl(SegmentedRaftLog.java:214)
        at org.apache.ratis.server.raftlog.RaftLogBase.open(RaftLogBase.java:251)
        at org.apache.ratis.server.impl.ServerState.initRaftLog(ServerState.java:239)
        at org.apache.ratis.server.impl.ServerState.initRaftLog(ServerState.java:220)
        at org.apache.ratis.server.impl.ServerState.lambda$new$5(ServerState.java:161)
        at org.apache.ratis.util.MemoizedSupplier.get(MemoizedSupplier.java:62)
        at org.apache.ratis.server.impl.ServerState.initialize(ServerState.java:177)
        at org.apache.ratis.server.impl.RaftServerImpl.start(RaftServerImpl.java:338)
        at org.apache.ratis.util.ConcurrentUtils.accept(ConcurrentUtils.java:188)

segment log:

 

 

The modification time of this segment log is greater than the modification time of the file with the larger index.

The file size of this file seems to be larger than the other files, but it's not as big as the other files.

Issue Links

• is related to: RATIS-1891 Gap between logs cause service startup failure
• is related to: RATIS-2177 Purge should delete segmentLog from small to large according to logIndex
• relates to: RATIS-1763 Gap between logs caused by purging order
• links to: GitHub Pull Request #919