[HDFS-11886] Ozone : Optimize putKey operation to be async and consensus

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: ozone
• Labels: OzonePostMerge
• Target Version/s: ozone
• Assignee: Weiwei Yang
• Reporter: Chen Liang
• Created: 25/May/17 22:56
• Updated: 06/Sep/17 22:22

Description

Ozone's putKey operations involve a couple steps:
1. KSM calls allocateBlock to SCM, writes this info to KSM's local metastore
2. allocatedBlock gets returned to client, client checks to see if container needs to be created on datanode, if yes, create the container
3. writes the data to container.
it is possible that 1 succeeded, but 2 or 3 failed, in this case there will be an entry in KSM's local metastore, but the key is actually nowhere to be found. We need to revert 1 is 2 or 3 failed in this case.

To resolve this, we need at least two things to be implemented first.
1. We need deleteKey() to be added KSM first.
2. We also need container reports to be implemented first such that SCM can track whether the container is actually added.

Issue Links

• is duplicated by: HDFS-12144 Ozone: KSM: Cleanup of keys in KSM for failed clients