[HDDS-1313] "ozone scmcli create" throws an exception

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Cannot Reproduce
• Affects Version/s: 0.4.0
• Component/s: Ozone CLI, SCM Client
• Target Version/s: 0.5.0
• Assignee: Siddharth Wagle
• Reporter: Vivek Ratnavel Subramanian
• Created: 20/Mar/19 17:36
• Updated: 09/May/19 05:50
• Resolved: 09/May/19 05:50

Description

The command "ozone scmcli create" throws an exception, but creates a container

bash-4.2$ ozone scmcli create
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.ratis.thirdparty.com.google.protobuf.UnsafeUtil (file:/opt/hadoop/share/ozone/lib/ratis-thirdparty-misc-0.2.0.jar) to field java.nio.Buffer.address
WARNING: Please consider reporting this to the maintainers of org.apache.ratis.thirdparty.com.google.protobuf.UnsafeUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Failed to execute command cmdType: CreateContainer
traceID: "61e16838-b411-4957-93d2-a8168ef93ea0"
containerID: 2
datanodeUuid: "e6701c27-596e-4c30-8d6b-f61eb8804140"
createContainer {
containerType: KeyValueContainer
}
on datanode e6701c27-596e-4c30-8d6b-f61eb8804140
java.util.concurrent.ExecutionException: org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException: UNKNOWN
at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)
at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999)
at org.apache.hadoop.hdds.scm.XceiverClientGrpc.sendCommandWithRetry(XceiverClientGrpc.java:251)
at org.apache.hadoop.hdds.scm.XceiverClientGrpc.sendCommand(XceiverClientGrpc.java:207)
at org.apache.hadoop.hdds.scm.storage.ContainerProtocolCalls.createContainer(ContainerProtocolCalls.java:419)
at org.apache.hadoop.hdds.scm.client.ContainerOperationClient.createContainer(ContainerOperationClient.java:117)
at org.apache.hadoop.hdds.scm.client.ContainerOperationClient.createContainer(ContainerOperationClient.java:97)
at org.apache.hadoop.hdds.scm.cli.container.CreateSubcommand.call(CreateSubcommand.java:59)
at org.apache.hadoop.hdds.scm.cli.container.CreateSubcommand.call(CreateSubcommand.java:37)
at picocli.CommandLine.execute(CommandLine.java:919)
at picocli.CommandLine.access$700(CommandLine.java:104)
at picocli.CommandLine$RunLast.handle(CommandLine.java:1083)
at picocli.CommandLine$RunLast.handle(CommandLine.java:1051)
at picocli.CommandLine$AbstractParseResultHandler.handleParseResult(CommandLine.java:959)
at picocli.CommandLine.parseWithHandlers(CommandLine.java:1242)
at picocli.CommandLine.parseWithHandler(CommandLine.java:1181)
at org.apache.hadoop.hdds.cli.GenericCli.execute(GenericCli.java:61)
at org.apache.hadoop.hdds.cli.GenericCli.run(GenericCli.java:52)
at org.apache.hadoop.hdds.scm.cli.SCMCLI.main(SCMCLI.java:109)
Caused by: org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException: UNKNOWN
at org.apache.ratis.thirdparty.io.grpc.Status.asRuntimeException(Status.java:526)
at org.apache.ratis.thirdparty.io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:434)
at org.apache.ratis.thirdparty.io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at org.apache.ratis.thirdparty.io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at org.apache.ratis.thirdparty.io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at org.apache.ratis.thirdparty.io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1$1.onClose(CensusStatsModule.java:678)
at org.apache.ratis.thirdparty.io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at org.apache.ratis.thirdparty.io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at org.apache.ratis.thirdparty.io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at org.apache.ratis.thirdparty.io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1$1.onClose(CensusTracingModule.java:397)
at org.apache.ratis.thirdparty.io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:459)
at org.apache.ratis.thirdparty.io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:63)
at org.apache.ratis.thirdparty.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:546)
at org.apache.ratis.thirdparty.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$600(ClientCallImpl.java:467)
at org.apache.ratis.thirdparty.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:584)
at org.apache.ratis.thirdparty.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at org.apache.ratis.thirdparty.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Failed to execute command cmdType: CreateContainer

 
bash-4.2$ ozone scmcli list -s 0 
{ "state" : "OPEN", "replicationFactor" : "ONE", "replicationType" : "STAND_ALONE", "usedBytes" : 0, "numberOfKeys" : 0, "lastUsed" : 416742426, "stateEnterTime" : 416432914, "owner" : "OZONE", "containerID" : 1, "deleteTransactionId" : 0, "sequenceId" : 0, "open" : true } 
{ "state" : "OPEN", "replicationFactor" : "ONE", "replicationType" : "STAND_ALONE", "usedBytes" : 0, "numberOfKeys" : 0, "lastUsed" : 416742426, "stateEnterTime" : 416526702, "owner" : "OZONE", "containerID" : 2, "deleteTransactionId" : 0, "sequenceId" : 0, "open" : true }