[DERBY-2852] Derby: Replication and Availability

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 10.2.2.0
• Component/s: Network Server, Services, SQL, Store
• Assignee: Unassigned
• Reporter: Egil Sørensen
• Created: 21/Jun/07 10:05
• Updated: 17/Aug/12 09:03
• Resolved: 17/Aug/12 09:03

Description

This paper and attached source code describes the work done to add hot standby replication functionality to the Apache Derby Database Management System.

By implementing a hot standby scheme in Apache Derby several features are added. The contents of the database is replicated at run time to another site providing online runtime backup. As the hot standby takes over on faults availability is added in that a client can connect to the hot standby after a crash. Thus the crash is masked from the clients. In addition to this, online upgrades of software and hardware can be done by taking down one database at the time. Then when
the upgrade is completed the upgraded server is synchronized and back online with no downtime.

A fully functional prototype of the Apache Derby hot standby scheme has been created in this project using logical logs, fail-fast takeovers and logical catchups after an internal up-to-crash recovery and reconnection. This project builds on the ideas that are presented in my earlier project: Derby: Write to Neighbor Mode.

To enable the features described above some changes have been made to the Derby base system. The following classes have been changed or added:

The Logical Log System
org.apache.derby.iapi.store.logical.log.LogicalImage
org.apache.derby.iapi.store.logical.log.LogicalLoggable
org.apache.derby.impl.store.logical.log.LogicalLog
org.apache.derby.impl.store.logical.log.LogicalLogger
org.apache.derby.impl.store.logical.log.LogicalLogRecord
org.apache.derby.impl.store.logical.opearations.LogicalAbortOperation
org.apache.derby.impl.store.logical.opearations.LogicalCommitOperation
org.apache.derby.impl.store.logical.opearations.LogicalDeleteOperation
org.apache.derby.impl.store.logical.opearations.LogicalImage
org.apache.derby.impl.store.logical.opearations.LogicalInsertOperation
org.apache.derby.impl.store.logical.opearations.LogicalOperation
org.apache.derby.impl.store.logical.opearations.LogicalOperationFactory
org.apache.derby.impl.store.logical.opearations.LogicalUpdateOperation

The SQL Layer
org.apache.impl.sql.execute.RowChangerImpl
org.apache.impl.sql.conn.GenericLanguageConnectionContext

The Communications Service
org.apache.derby.impl.services.net.LogicalServer
org.apache.derby.impl.services.net.LogicalClient
org.apache.derby.impl.services.net.NetworkPayload
org.apache.derby.impl.services.net.receiver.LogReceiver
org.apache.derby.impl.services.net.shipper.LogShipper
org.apache.derby.impl.services.net.shipper.LogicalCatchUp

The Hot Standby Service
org.apache.derby.impl.services.hotstandby.HotStandbyController
org.apache.derby.impl.services.hotstandby.Database

The Network Server
org.apache.derby.impl.drda.NetworkServerControlImpl
org.apache.derby.impl.drda.HotStandbyPoller
org.apache.derby.impl.drda.net.LogicalClient
org.apache.derby.imp.drda.net.LogicalServer

See the added paper for more thorough documentation on design choices and implementation along with some benchmarking results

Issue Links

• relates to: DERBY-2872 Add Replication functionality to Derby