Bulk Operation

HDDS-2823 [Doc] Update OM HA Setup Doc

HDDS-2823 terminate om if statemachine is shut down by ratis

HDDS-2823 use `Fileutils.move` instead of `Files.move` when installing snapshot

HDDS-2823 Add ratis metric for scm

HDDS-2823 For any IOexception from @Replicated method we should throw it

HDDS-2823 remove scm from SCM HA group

HDDS-2823 [SCM-HA] SCM start failed with PipelineNotFoundException

HDDS-2823 [SCM-HA] SCM start failed with PipelineNotFoundException

HDDS-2823 Add reinitialize() for SequenceIdGenerator.

HDDS-2823 Handle unsecure cluster convert to secure cluster for SCM

HDDS-2823 Fix fall back of config in SCM HA Cluster

HDDS-2823 SCM HA: Continuous PipelineNotFoundException seen in SCM log

HDDS-2823 Wait for ever to obtain CA list which is needed during OM/DN startup

HDDS-2823 SCM UI should have leader/follower and Primordial SCM information

HDDS-2823 Change default grpc and ratis ports for scm ha

HDDS-2823 SCM subsequent init failed when previous scm init failed

HDDS-2823 ozone freon randomkeys failed after leader SCM node is down

HDDS-2823 Make admin check work for SCM HA cluster

HDDS-2823 Use scm#checkLeader before processing client requests

HDDS-2823 Fix scm roles command if one of the host is unresolvable

HDDS-2823 If primordial SCM id is set, a non-HA cluster can not be initialized.

HDDS-2823 For AccessControlException do not perform failover

HDDS-2823 Divide snapshot related work into notifyInstallSnapshotFromLeader and reinitialize for SCMStateMachine.

HDDS-2823 Fix Suggested leader in Client

HDDS-2823 Fix Install Snapshot Mechanism in SCMStateMachine

HDDS-2823 make Decommission work under SCM HA.

HDDS-2823 [SCM HA Security] Enable s3 test suite for ozone-secure-ha

HDDS-2823 [SCM HA Security] NPE during secure SCM initialization with HA code updated to an already existing cluster

HDDS-2823 [SCM HA Security] Remove code of not starting ozone services when Security is enabled on SCM HA cluster

HDDS-2823 [SCM HA Security] Fix duration of sub-ca certs

HDDS-2823 [SCM HA Security] Make InterSCM grpc channel secure

HDDS-2823 Merge SCM HA configs to ScmConfigKeys

HDDS-2823 [SCM HA Security] Handle leader changes between SCMInfo and getSCMSigned Cert in OM

HDDS-2823 Ensure failover to suggested leader if any for NotLeaderException

HDDS-2823 SCM may not be able to know full port list of Datanode after Datanode is started.

HDDS-2823 [SCM HA Security] Handle leader changes during bootstrap

HDDS-2823 SCM get roles command should provide Ratis Leader/Follower information.

HDDS-2823 localId is not consistent across SCMs when setup a multi node SCM HA cluster.

HDDS-2823 [SCM HA Security] Make changes required for ratis enabled with new model of RootCA/subCA

HDDS-2823 [SCM HA Security] generate certserialID in distributed sequence

HDDS-2823 [SCM HA Security] Make storeValidCertificate method idempotent

HDDS-2823 [SCM HA Security] When Ratis is enabled, SCM secure cluster is not working

HDDS-2823 Solve intellj warnings on DBTransactionBuffer.

HDDS-2823 Remove SequenceIdGenerator#StateManagerImpl

HDDS-2823 [SCM HA Security] Ozone services should be disabled in SCM HA enabled and security enabled cluster

HDDS-2823 Back-port HDDS-4911 (List container by container state) to ContainerManagerV2

HDDS-2823 Adapt admincli tests for SCM HA

HDDS-2823 Fix flaky test TestSCMInstallSnapshotWithHA#testInstallCorruptedCheckpointFailure

HDDS-2823 [SCM HA Security] Make CertStore DB updates for StoreValidateCertificate go via Ratis

HDDS-2823 [SCM HA Security] Implement listCAs and getRootCA API

HDDS-2823 Return with exit code 0 in case of optional scm bootstrap/init

HDDS-2823 Provide example k8s files to run full HA Ozone

HDDS-2823 NullPointerException during SCM init

HDDS-2823 [Doc] Add SCM HA Setup Doc

HDDS-2823 refactor code in SCMStateMachine.

HDDS-2823 [SCM HA Security] Integrate CertClient

HDDS-2823 Support inline upgrade from containerId, delTxnId, localId to SequenceIdGenerator.

HDDS-2823 Add SCM HA to Chaos tests

HDDS-2823 [SCM HA Security] Create SCM Cert Client and change DefaultCA to allow self signed and intermediary

HDDS-2823 Need a tool to upgrade current non-HA SCM node to single node HA cluster

HDDS-2823 Use PipelineManagerV2Impl in Recon and enable ignored Recon test cases.

HDDS-2823 SCM Ratis enable/disable switch

HDDS-2823 Fix and enable TestEndpoints.java

HDDS-2823 Fix and enable TestReconTasks

HDDS-2823 Fix removing local SCM when submitting request to other SCM.

HDDS-2823 Make SCM ratis server spin up time during initialization configurable

HDDS-2823 [SCM HA Security] Add failover proxy to SCM Security Server Protocol

HDDS-2823 [SCM HA Security] Implement listCertificates based on role

HDDS-2823 Datanode with scmID format should work with clusterID directory format

HDDS-2823 Provide docker-compose for SCM HA

HDDS-2823 [SCM HA Security] Implement generate SCM certificate

HDDS-2823 Reopen replication/wait.robot added by HDDS-4834

HDDS-2823 Make changes required for SCM admin commands to work with SCM HA

HDDS-2823 Use SCM service ID in finding SCM Datanode address.

HDDS-2823 Rename MiniOzoneHACluster to MiniOzoneOMHACluster

HDDS-2823 Make SCM Generic config support HA Style

HDDS-2823 Implement scm --bootstrap command

HDDS-2823 Use SCM service ID in SCMBlockClient and SCM Client

HDDS-2823 Add multiple SCM nodes to MiniOzoneCluster

HDDS-2823 Move Ratis group creation to scm --init phase

HDDS-2823 Use OM style config to construct RaftGroup and initialize Raft Servers

HDDS-2823 Integrate DeleteBlockLog with PartialTableCache

HDDS-2823 Add install checkpoint in SCMStateMachine

HDDS-2823 Fix misc acceptance test: List pipelines on unknown host

HDDS-2823 Fix TestReconContainerManager after merge master to HDDS-2823

HDDS-2823 Fix TestContainerEndpoint after merging master to HDDS-2823.

HDDS-2823 Fix findbugs issues after HDDS-2195

HDDS-2823 Disable Prevote in Ratis in SCM HA by default

HDDS-2823 Merge SCMRatisSnapshotInfo and OMRatisSnapshotInfo into a single class

HDDS-2823 Temporarily ignore failing Recon tests

HDDS-2823 Add transactionId into deletingTxIDs when remove it from DB

HDDS-2823 Add functionality to transfer Rocks db checkpoint from leader to follower

HDDS-2823 Implement increment count optimization in DeletedBlockLog V2

HDDS-2823 activatePipeline/deactivatePipeline in PipelineManagerV2Impl should acquire lock before calling StateManager#updatePipelineState.

HDDS-2823 Add STOP state to SCMService.

HDDS-2823 Admin command should take effect on all SCM instance

HDDS-2823 Bootstrap new SCM node

HDDS-2823 Replace UniqueID by the Distributed Sequence ID Generator

HDDS-2823 Support encode and decode ArrayList and Long

HDDS-2823 Use applyTransactionSerial instead of applyTransaction

HDDS-2823 Merge OMTransactionInfo with SCMTransactionInfo

HDDS-2823 Should disallow log purge before installSnapshot is implemented

HDDS-2823 Implement Distributed Sequence ID Generator

HDDS-2823 Ratis Snapshot should be loaded from the confg

HDDS-2823 Change default SCM snapshot frequency to a lower value

HDDS-2823 Implement DB buffer in MockHAManager

HDDS-2823 Fix SCMHAManagerImpl#isLeader after RATIS-1227

HDDS-2823 Add term into SetNodeOperationalStateCommand.

HDDS-2823 Solve deadlock triggered by PipelineActionHandler.

HDDS-2823 min/max election timeout of SCMRatisServer is not set properly.

HDDS-2823 Fix set configs in SCMHAConfigration

HDDS-2823 Use singe server raft cluster in MiniOzoneCluster.

HDDS-2823 Use OM style Configuration to initialize SCM HA

HDDS-2823 Handle potential data loss during ReplicationManager.handleOverReplicatedContainer()

HDDS-2823 Refactor SCMHAManager and SCMRatisServer with RaftServer.Division

HDDS-2823 SCMContext

HDDS-2823 SCMContext Phase 1 - Raft Related Info

HDDS-2823 Add ReadWriteLock into PipelineStateManagerV2Impl to protect contentions between RaftServer and PipelineManager

HDDS-2823 Remove checkLeader in PipelineManager.

HDDS-2823 Need throw exception to trigger FailoverProxyProvider of SCM client to work

HDDS-2823 Avoid rewriting pipeline information during PipelineStateManagerV2Impl initialization

HDDS-2823 Update pipeline db when pipeline state is changed

HDDS-2823 Fix SCMHAManager#getPeerIdFromRoleInfo

HDDS-2823 Fix TestMiniOzoneHACluster.testGetOMLeader()

HDDS-2823 Fix CI and test failures after force push on 2020/10/26

HDDS-2823 SCMBlockLocationFailoverProxyProvider should use ScmBlockLocationProtocolPB.class in RPC.setProtocolEngine

HDDS-2823 SCM ServiceManager

HDDS-2823 Backport updates from ContainerManager(V1)

HDDS-2823 Backport updates from PipelineManager(V1)

HDDS-2823 Use suggestedLeader for SCM failover proxy performing failover

HDDS-2823 Fix Recon after HDDS-4133

HDDS-2823 enable SCM Raft Group based on config ozone.scm.names

HDDS-2823 Add failover proxy for SCM container client

HDDS-2823 Use new ContainerManager in SCM

HDDS-2823 Switch to ContainerManagerV2

HDDS-2823 remove the 1st edition of RatisServer of SCM HA which is copied from OM HA

HDDS-2823 RATIS ONE Pipeline is closed but not removed when a datanode goes stale

HDDS-2823 Pipeline is not removed when a datanode goes stale

HDDS-2823 CLI command to show current SCM leader and follower status

HDDS-2823 replace scmID with clusterID for container and volume at Datanode side

HDDS-2823 update RATIS version from 1.0.0 to 1.1.0-85281b2-SNAPSHOT

HDDS-2823 SCMStateMachine::applyTransaction() should not invoke TransactionContext.getClientRequest()

HDDS-2823 DN can distinguish SCMCommand from stale leader SCM

HDDS-2823 Use getRoleInfoProto() in isLeader check

HDDS-2823 Implement container related operations in ContainerManagerImpl

HDDS-2823 Stop BackgroundPipelineCreator when PipelineManager is closed

HDDS-2823 Handle stale leader issue

HDDS-2823 Add isLeader check for SCM state updates

HDDS-2823 PipelineStateManagerV2Impl#removePipeline will remove pipeline from db in case of failure

HDDS-2823 Handle inner classes in SCMRatisRequest and SCMRatisResponse

HDDS-2823 Switch to PipelineStateManagerV2 and put PipelineFactory in PipelineManager

HDDS-2823 Add tests for replication annotation

HDDS-2823 Add unit tests for new PipelineManager interface

HDDS-2823 Handle events fired from PipelineManager to close container

HDDS-2823 Introduce SCMStateMachineHandler marker interface

HDDS-2823 decouple finalize and destroy pipeline

HDDS-2823 Add Snapshot into new SCMRatisServer and SCMStateMachine

HDDS-2823 Add unit test for SCMRatisResponse

HDDS-2823 Add unit test for SCMRatisRequest

HDDS-2823 Refactor configuration in SCMRatisServer to Java-based configuration

HDDS-2823 SCM Invoke Handler for Ratis calls

HDDS-2823 acceptance test for SCM HA

HDDS-2823 Allow Enabling Purge SCM Ratis log

HDDS-2823 Load Snapshot info upon SCM Ratis starts

HDDS-2823 Implement Ratis Snapshots on SCM

HDDS-2823 Provide docker-compose for SCM HA

HDDS-2823 Make sure AllocateBlock can only be executed on leader SCM

HDDS-2823 DeleteBlock via Ratis in SCM HA

HDDS-2823 Replication can only be executed on leader

HDDS-2823 Bootstrap SCM HA Security

HDDS-2823 SafeMode exit rule for all SCMs

HDDS-2823 Handle NodeReport from DN to SCMs

HDDS-2823 Handle PipelineAction and OpenPipline from DN to SCM

HDDS-2823 Handle pipeline reports

HDDS-2823 BackgroundPipelineCreator can only serve leader

HDDS-2823 New PipelineManager interface to persist to RatisServer

HDDS-2823 Handle ContainerAction and CloseContainer

HDDS-2823 Handle ContainerReport and IncrementalContainerReport

HDDS-2823 Handle AllocateContainer operation for HA

HDDS-2823 Switch current pipeline interface to the new Replication based interface to write to Ratis

HDDS-2823 SCM needs to replay RaftLog for recovery

HDDS-2823 Add failover proxy to SCM block protocol

HDDS-2823 SCM StateMachine

HDDS-2823 Introduce generic SCMRatisRequest and SCMRatisResponse

HDDS-2823 Standalone SCM RatisServer