Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-2823 SCM HA Support
  3. HDDS-4136

Design for Error/Exception handling in state update for container/pipeline V2

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotAdd voteVotersWatch issueWatchersConvert to IssueLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: SCM
    • Labels:
      None

      Description

      I have a concern about how to handle exceptions occurred in writing RocksDB for container V2, such as allocateContainer, deleteContainer and updateContainerState.

      For non-HA case, allocateContainer reverts the memory state changes if meet IOException for db operations. deleteContainer and updateContainerState just throw out the IOException and leave the memory state in an inconsistency state.

      After we enable SCM-HA, if leader SCM succeed the operation, meanwhile any follower SCM fails due to db exception, what can we do to ensure that states of leader and followers won't diverge, a.k.a. ensure the replicated StateMachine for leader and followers ?

      We have to ensure Atomicity of ACID for state update: If any exception occurred, SCM (no matter leader or follower) should throw exception and keep states unchanged. No partial change is allowed so that leader SCM can safely revert the state change for the whole raft groups.

      Above analysis also applies to pipeline V2 and other issues besides disk failure.

        Attachments

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

              Dates

              • Created:
                Updated:

                Issue deployment