[HDDS-461] Container remains in CLOSING state in SCM forever

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.2.1
• Fix Version/s: 0.2.1
• Component/s: SCM
• Target Version/s: 0.2.1
• Assignee: Shashikant Banerjee
• Reporter: Nilotpal Nandi
• Created: 14/Sep/18 10:28
• Updated: 04/Apr/19 09:01
• Resolved: 19/Sep/18 16:16

Description

Container id # 13's state is not changing from CLOSING to CLOSED.

[root@ctr-e138-1518143905142-459606-01-000002 bin]# ./ozone scmcli info 13
raft.rpc.type = GRPC (default)
raft.grpc.message.size.max = 33554432 (custom)
raft.client.rpc.retryInterval = 300 ms (default)
raft.client.async.outstanding-requests.max = 100 (default)
raft.client.async.scheduler-threads = 3 (default)
raft.grpc.flow.control.window = 1MB (=1048576) (default)
raft.grpc.message.size.max = 33554432 (custom)
raft.client.rpc.request.timeout = 3000 ms (default)
Container id: 13
Container State: OPEN
Container Path: /tmp/hadoop-root/dfs/data/hdds/de0a9e01-4a12-40e3-b567-51b9bd83248e/current/containerDir0/13/metadata
Container Metadata:
LeaderID: ctr-e138-1518143905142-459606-01-000003.hwx.site
Datanodes: [ctr-e138-1518143905142-459606-01-000007.hwx.site,ctr-e138-1518143905142-459606-01-000008.hwx.site,ctr-e138-1518143905142-459606-01-000003.hwx.site]

 

snippet of scmcli list :

{
 "state" : "CLOSING",
 "replicationFactor" : "THREE",
 "replicationType" : "RATIS",
 "allocatedBytes" : 4831838208,
 "usedBytes" : 4831838208,
 "numberOfKeys" : 0,
 "lastUsed" : 4391827471,
 "stateEnterTime" : 5435591457,
 "owner" : "f8332db1-b8b1-4077-a9ea-097033d074b7",
 "containerID" : 13,
 "deleteTransactionId" : 0,
 "containerOpen" : true
}

Issue Links

• contains: HDDS-494 client fails to write when all the containers are closed through scmcli