[RATIS-516] IllegalStateException on GrpcLogAppender#AppendLogResponseHandler processing

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 0.4.0
• Component/s: server
• Labels: MiniOzoneChaosCluster
• Target Version/s: 0.4.0
• Assignee: Tsz-wo Sze
• Reporter: Mukul Kumar Singh
• Created: 07/Apr/19 01:58
• Updated: 14/Jul/19 15:31
• Resolved: 14/Jul/19 15:31

Description

The following error is observed on Ozone Dataodes while running IO.

2019-04-05 11:12:46,081 ERROR server.GrpcLogAppender (GrpcLogAppender.java:onNext(226)) - Failed onNext serverReply {
  requestorId: "ba438b75-8549-4cbe-ab0d-7de0e1ed173d"
  replyId: "1cede484-3dc6-4518-9587-1c6f8e7747e7"
  raftGroupId {
    id: "\\\201\033c\350\373B9\263\335\214\357\274`\315\215"
  }
  callId: 1305
  success: true
}
term: 2
nextIndex: 3171
followerCommit: 3165

java.lang.IllegalStateException: reply's next index is 3171, request's previous is term: 2
index: 3169

        at org.apache.ratis.util.Preconditions.assertTrue(Preconditions.java:60)
        at org.apache.ratis.grpc.server.GrpcLogAppender.onSuccess(GrpcLogAppender.java:294)
        at org.apache.ratis.grpc.server.GrpcLogAppender$AppendLogResponseHandler.onNextImpl(GrpcLogAppender.java:239)
        at org.apache.ratis.grpc.server.GrpcLogAppender$AppendLogResponseHandler.onNext(GrpcLogAppender.java:224)
        at org.apache.ratis.grpc.server.GrpcLogAppender$AppendLogResponseHandler.onNext(GrpcLogAppender.java:206)
        at org.apache.ratis.thirdparty.io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onMessage(ClientCalls.java:421)
        at org.apache.ratis.thirdparty.io.grpc.ForwardingClientCallListener.onMessage(ForwardingClientCallListener.java:33)
        at org.apache.ratis.thirdparty.io.grpc.ForwardingClientCallListener.onMessage(ForwardingClientCallListener.java:33)
        at org.apache.ratis.thirdparty.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1MessagesAvailable.runInContext(ClientCallImpl.java:519)
        at org.apache.ratis.thirdparty.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
        at org.apache.ratis.thirdparty.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Issue Links

• is duplicated by: RATIS-515 IllegalStateException while updating matchIndex
• is part of: RATIS-515 IllegalStateException while updating matchIndex