Uploaded image for project: 'Hadoop Distributed Data Store'
  1. Hadoop Distributed Data Store
  2. HDDS-1940

Closing open container via scmcli gives false error message

    XMLWordPrintableJSON

    Details

    • Target Version/s:

      Description

      scmcli close prints an error message about invalid state transition after it had successfully closed the container.

      CLI
      $ ozone scmcli info 2
      ...
      Container State: OPEN
      ...
      
      $ ozone scmcli close 2
      ...
      client-09830A377AA9->f27bf787-8711-41d4-b0fd-3ef50b5c076f: receive RaftClientReply:client-09830A377AA9->f27bf787-8711-41d4-b0fd-3ef50b5c076f@group-7831D6F2EF1B, cid=0, SUCCESS, logIndex=11, commits[f27bf787-8711-41d4-b0fd-3ef50b5c076f:c12, 37ba33fe-c9ed-4ac2-a6e5-57ce658168b4:c11, feb68ba4-0a8a-4eda-9915-7dc090e5f46c:c11]
      Failed to update container state #2, reason: invalid state transition from state: CLOSED upon event: CLOSE.
      
      $ ozone scmcli info 2
      ...
      Container State: CLOSED
      ...
      
      logs
      scm_1  | 2019-08-09 15:15:01 [IPC Server handler 1 on 9860] INFO  SCMClientProtocolServer:366 - Object type container id 1 op close new stage begin
      dn3_1  | 2019-08-09 15:15:02 [RatisApplyTransactionExecutor 1] INFO  Container:356 - Container 1 is closed with bcsId 3.
      dn1_1  | 2019-08-09 15:15:02 [RatisApplyTransactionExecutor 1] INFO  Container:356 - Container 1 is closed with bcsId 3.
      scm_1  | 2019-08-09 15:15:02 [EventQueue-IncrementalContainerReportForIncrementalContainerReportHandler] INFO  IncrementalContainerReportHandler:176 - Moving container #1 to CLOSED state, datanode feb68ba4-0a8a-4eda-9915-7dc090e5f46c{ip: 10.5.1.6, host: ozone-static_dn3_1.ozone-static_net, networkLocation: /default-rack, certSerialId: null} reported CLOSED replica.
      dn2_1  | 2019-08-09 15:15:02 [RatisApplyTransactionExecutor 1] INFO  Container:356 - Container 1 is closed with bcsId 3.
      scm_1  | 2019-08-09 15:15:02 [IPC Server handler 3 on 9860] INFO  SCMClientProtocolServer:366 - Object type container id 1 op close new stage complete
      scm_1  | 2019-08-09 15:15:02 [IPC Server handler 3 on 9860] ERROR ContainerStateManager:335 - Failed to update container state #1, reason: invalid state transition from state: CLOSED upon event: CLOSE.
      scm_1  | 2019-08-09 15:15:02 [IPC Server handler 3 on 9860] INFO  Server:2726 - IPC Server handler 3 on 9860, call Call#3 Retry#0 org.apache.hadoop.hdds.scm.protocol.StorageContainerLocationProtocol.notifyObjectStageChange from 10.5.0.71:57746
      scm_1  | org.apache.hadoop.hdds.scm.exceptions.SCMException: Failed to update container state #1, reason: invalid state transition from state: CLOSED upon event: CLOSE.
      scm_1  | 	at org.apache.hadoop.hdds.scm.container.ContainerStateManager.updateContainerState(ContainerStateManager.java:336)
      scm_1  | 	at org.apache.hadoop.hdds.scm.container.SCMContainerManager.updateContainerState(SCMContainerManager.java:312)
      scm_1  | 	at org.apache.hadoop.hdds.scm.server.SCMClientProtocolServer.notifyObjectStageChange(SCMClientProtocolServer.java:379)
      scm_1  | 	at org.apache.hadoop.ozone.protocolPB.StorageContainerLocationProtocolServerSideTranslatorPB.notifyObjectStageChange(StorageContainerLocationProtocolServerSideTranslatorPB.java:219)
      scm_1  | 	at org.apache.hadoop.hdds.protocol.proto.StorageContainerLocationProtocolProtos$StorageContainerLocationProtocolService$2.callBlockingMethod(StorageContainerLocationProtocolProtos.java:16398)
      scm_1  | 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524)
      scm_1  | 	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1025)
      scm_1  | 	at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:876)
      scm_1  | 	at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:822)
      scm_1  | 	at java.base/java.security.AccessController.doPrivileged(Native Method)
      scm_1  | 	at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
      scm_1  | 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
      scm_1  | 	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2682)
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                nanda Nanda kumar
                Reporter:
                adoroszlai Attila Doroszlai
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m