Details
-
Bug
-
Status: Resolved
-
Blocker
-
Resolution: Fixed
-
2.5.0
-
None
-
Reviewed
Description
Concerns about SimpleRpcServer are not new, and not new to 2.5. @chenxu noticed a problem on HBASE-23917 back in 2020. After some simple evaluations it seems quite broken.
When I run an async version of ITLCC against a 2.5.0 cluster configured with hbase.rpc.server.impl=SimpleRpcServer, the client almost immediately stalls because there are too many in flight requests. The logic to pause with too many in flight requests is my own. That's not important. Looking at the server logs it is apparent that SimpleRpcServer is quite broken. Handlers suffer frequent protobuf parse errors and do not properly return responses to the client. This is what stalls my test client. Rather quickly all available request slots are full of requests that will have to time out on the client side.
Exceptions have three patterns but they all have in common SimpleServerRpcConnection#process. It seems likely the root cause is mismatched expectations or bugs in connection buffer handling in SimpleRpcServer/SimpleServerRpcConnection versus downstream classes that process and parse the buffers. It also seems likely that changes were made to downstream classes like ServerRpcConnection expecting NettyRpcServer's particulars without updating SimpleServerRpcConnection and/or SimpleRpcServer. That said, this is just a superficial analysis.
1) "Protocol message end-group tag did not match expected tag"
2022-06-07T16:44:04,625 WARN [Reader=5,bindAddress=buildbox.localdomain,port=8120] ipc.RpcServer: /127.0.1.1:8120 is unable to read call parameter from client 127.0.0.1 org.apache.hbase.thirdparty.com.google.protobuf.InvalidProtocolBufferException: Protocol message end-group tag did not match expected tag. at org.apache.hbase.thirdparty.com.google.protobuf.InvalidProtocolBufferException.invalidEndTag(InvalidProtocolBufferException.java:129) ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0] at org.apache.hbase.thirdparty.com.google.protobuf.CodedInputStream$ByteInputDecoder.checkLastTagWas(CodedInputStream.java:4034) ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0] at org.apache.hbase.thirdparty.com.google.protobuf.CodedInputStream$ByteInputDecoder.readMessage(CodedInputStream.java:4275) ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue.<init>(ClientProtos.java:10520) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue.<init>(ClientProtos.java:10464) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$1.parsePartialFrom(ClientProtos.java:12251) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$1.parsePartialFrom(ClientProtos.java:12245) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hbase.thirdparty.com.google.protobuf.CodedInputStream$ByteInputDecoder.readMessage(CodedInputStream.java:4274) ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto.<init>(ClientProtos.java:9981) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto.<init>(ClientProtos.java:9910) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$1.parsePartialFrom(ClientProtos.java:14097) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$1.parsePartialFrom(ClientProtos.java:14091) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hbase.thirdparty.com.google.protobuf.CodedInputStream$ByteInputDecoder.readMessage(CodedInputStream.java:4274) ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest.<init>(ClientProtos.java:14251) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest.<init>(ClientProtos.java:14190) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest$1.parsePartialFrom(ClientProtos.java:15304) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest$1.parsePartialFrom(ClientProtos.java:15298) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest$Builder.mergeFrom(ClientProtos.java:14860) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest$Builder.mergeFrom(ClientProtos.java:14651) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hbase.thirdparty.com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:420) ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0] at org.apache.hbase.thirdparty.com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:317) ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0] at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.mergeFrom(ProtobufUtil.java:2638) ~[hbase-client-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.ipc.ServerRpcConnection.processRequest(ServerRpcConnection.java:644) ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.ipc.ServerRpcConnection.processOneRpc(ServerRpcConnection.java:444) ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.ipc.SimpleServerRpcConnection.process(SimpleServerRpcConnection.java:285) ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.ipc.SimpleServerRpcConnection.readAndProcess(SimpleServerRpcConnection.java:251) ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener.doRead(SimpleRpcServer.java:318) ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.doRunLoop(SimpleRpcServer.java:180) ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.run(SimpleRpcServer.java:153) ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
2) "Protocol message tag had invalid wire type."
2022-06-07T16:44:04,705 WARN [Reader=6,bindAddress=buildbox.localdomain,port=8120] ipc.RpcServer: /127.0.1.1:8120 is unable to read call parameter from client 127.0.0.1 org.apache.hbase.thirdparty.com.google.protobuf.InvalidProtocolBufferException$InvalidWireTypeException: Protocol message tag had invalid wire type. at org.apache.hbase.thirdparty.com.google.protobuf.InvalidProtocolBufferException.invalidWireType(InvalidProtocolBufferException.java:134) ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0] at org.apache.hbase.thirdparty.com.google.protobuf.UnknownFieldSet$Builder.mergeFieldFrom(UnknownFieldSet.java:527) ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0] at org.apache.hbase.thirdparty.com.google.protobuf.GeneratedMessageV3.parseUnknownField(GeneratedMessageV3.java:320) ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$QualifierValue.<init>(ClientProtos.java:10700) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$QualifierValue.<init>(ClientProtos.java:10620) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$QualifierValue$1.parsePartialFrom(ClientProtos.java:11481) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$QualifierValue$1.parsePartialFrom(ClientProtos.java:11475) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hbase.thirdparty.com.google.protobuf.CodedInputStream$ByteInputDecoder.readMessage(CodedInputStream.java:4274) ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue.<init>(ClientProtos.java:10520) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue.<init>(ClientProtos.java:10464) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$1.parsePartialFrom(ClientProtos.java:12251) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$1.parsePartialFrom(ClientProtos.java:12245) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hbase.thirdparty.com.google.protobuf.CodedInputStream$ByteInputDecoder.readMessage(CodedInputStream.java:4274) ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto.<init>(ClientProtos.java:9981) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto.<init>(ClientProtos.java:9910) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$1.parsePartialFrom(ClientProtos.java:14097) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$1.parsePartialFrom(ClientProtos.java:14091) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hbase.thirdparty.com.google.protobuf.CodedInputStream$ByteInputDecoder.readMessage(CodedInputStream.java:4274) ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest.<init>(ClientProtos.java:14251) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest.<init>(ClientProtos.java:14190) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest$1.parsePartialFrom(ClientProtos.java:15304) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest$1.parsePartialFrom(ClientProtos.java:15298) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest$Builder.mergeFrom(ClientProtos.java:14860) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest$Builder.mergeFrom(ClientProtos.java:14651) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hbase.thirdparty.com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:420) ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0] at org.apache.hbase.thirdparty.com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:317) ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0] at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.mergeFrom(ProtobufUtil.java:2638) ~[hbase-client-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.ipc.ServerRpcConnection.processRequest(ServerRpcConnection.java:644) ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.ipc.ServerRpcConnection.processOneRpc(ServerRpcConnection.java:444) ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.ipc.SimpleServerRpcConnection.process(SimpleServerRpcConnection.java:285) ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.ipc.SimpleServerRpcConnection.readAndProcess(SimpleServerRpcConnection.java:251) ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener.doRead(SimpleRpcServer.java:318) ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.doRunLoop(SimpleRpcServer.java:180) ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.run(SimpleRpcServer.java:153) ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?]
3) "While parsing a protocol message, the input ended unexpectedly in the middle of a field."
2022-06-07T16:44:04,885 WARN [Reader=9,bindAddress=buildbox.localdomain,port=8120] ipc.RpcServer: /127.0.1.1:8120 is unable to read call parameter from client 127.0.0.1 org.apache.hbase.thirdparty.com.google.protobuf.InvalidProtocolBufferException: While parsing a protocol message, the input ended unexpectedly in the middle of a field. This could mean either that the input has been truncated or that an embedded message misreported its own length. at org.apache.hbase.thirdparty.com.google.protobuf.InvalidProtocolBufferException.truncatedMessage(InvalidProtocolBufferException.java:107) ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0] at org.apache.hbase.thirdparty.com.google.protobuf.CodedInputStream$ByteInputDecoder.readRawLittleEndian64(CodedInputStream.java:4478) ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0] at org.apache.hbase.thirdparty.com.google.protobuf.CodedInputStream$ByteInputDecoder.readFixed64(CodedInputStream.java:4167) ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0] at org.apache.hbase.thirdparty.com.google.protobuf.UnknownFieldSet$Builder.mergeFieldFrom(UnknownFieldSet.java:511) ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0] at org.apache.hbase.thirdparty.com.google.protobuf.GeneratedMessageV3.parseUnknownField(GeneratedMessageV3.java:320) ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$QualifierValue.<init>(ClientProtos.java:10700) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$QualifierValue.<init>(ClientProtos.java:10620) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$QualifierValue$1.parsePartialFrom(ClientProtos.java:11481) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$QualifierValue$1.parsePartialFrom(ClientProtos.java:11475) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hbase.thirdparty.com.google.protobuf.CodedInputStream$ByteInputDecoder.readMessage(CodedInputStream.java:4274) ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue.<init>(ClientProtos.java:10520) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue.<init>(ClientProtos.java:10464) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$1.parsePartialFrom(ClientProtos.java:12251) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$ColumnValue$1.parsePartialFrom(ClientProtos.java:12245) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hbase.thirdparty.com.google.protobuf.CodedInputStream$ByteInputDecoder.readMessage(CodedInputStream.java:4274) ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto.<init>(ClientProtos.java:9981) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto.<init>(ClientProtos.java:9910) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$1.parsePartialFrom(ClientProtos.java:14097) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutationProto$1.parsePartialFrom(ClientProtos.java:14091) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hbase.thirdparty.com.google.protobuf.CodedInputStream$ByteInputDecoder.readMessage(CodedInputStream.java:4274) ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest.<init>(ClientProtos.java:14251) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest.<init>(ClientProtos.java:14190) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest$1.parsePartialFrom(ClientProtos.java:15304) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest$1.parsePartialFrom(ClientProtos.java:15298) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest$Builder.mergeFrom(ClientProtos.java:14860) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MutateRequest$Builder.mergeFrom(ClientProtos.java:14651) ~[hbase-protocol-shaded-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hbase.thirdparty.com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:420) ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0] at org.apache.hbase.thirdparty.com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:317) ~[hbase-shaded-protobuf-4.1.0.jar:4.1.0] at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.mergeFrom(ProtobufUtil.java:2638) ~[hbase-client-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.ipc.ServerRpcConnection.processRequest(ServerRpcConnection.java:644) ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.ipc.ServerRpcConnection.processOneRpc(ServerRpcConnection.java:444) ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.ipc.SimpleServerRpcConnection.process(SimpleServerRpcConnection.java:285) ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.ipc.SimpleServerRpcConnection.readAndProcess(SimpleServerRpcConnection.java:251) ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener.doRead(SimpleRpcServer.java:318) ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.doRunLoop(SimpleRpcServer.java:180) ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.run(SimpleRpcServer.java:153) ~[hbase-server-2.5.1-SNAPSHOT.jar:2.5.1-SNAPSHOT] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?]
Attachments
Attachments
Issue Links
- is blocked by
-
-
- Resolved
-
- relates to
-
-
- Resolved
-
- links to
There are no Sub-Tasks for this issue.