[OAK-1750] Adapt write concern dynamically to manage replication lag

• Type: Wish
• Status: Resolved
• Priority: Major
• Resolution: Later
• Component/s: mongomk
• Assignee: Chetan Mehrotra
• Reporter: Chetan Mehrotra
• Created: 21/Apr/14 16:55
• Updated: 08/Oct/19 15:21
• Resolved: 03/Nov/16 06:21

Description

Mongo provides write concern with varying level of durability and performance characteristics [1]. When using replica sets its possible that replica member lag behind in synching up the changes from primary. This introduces replication lag which can increase if the write rate is quite high.

One of the recommended ways [2], [3] to mitigate such scenarios is to change the write concern to higher values (journal ack, replica ack) to as to slow down writes on primary

In MongoDocumentStore it can be supported in following way

1. Have a periodic job which obtains current replSetGetStatus [4]
2. Determine the potential lag from the command result
3. if the lag is significant then change the write concern used in MongoDocumentStore to Syned, Journal Ack, Replica ack etc

[1] http://docs.mongodb.org/manual/core/write-concern/
[2] http://docs.mongodb.org/manual/tutorial/troubleshoot-replica-sets/#check-the-replication-lag
[3] http://blog.mongolab.com/2013/03/replication-lag-the-facts-of-life/
[4] http://docs.mongodb.org/manual/reference/command/replSetGetStatus/#dbcmd.replSetGetStatus

Issue Links

• is related to: OAK-2106 Optimize reads from secondaries