[ATLAS-2738] Export Process: Add Support for Incremental Export

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.8.3
• Fix Version/s: 0.8.3, 1.2.0, 2.0.0
• Component/s: atlas-core
• Assignee: Ashutosh Mestry
• Reporter: Ashutosh Mestry
• Due: 08/Jun/18
• Created: 04/Jun/18 21:25
• Updated: 26/Sep/19 00:38
• Resolved: 16/Aug/18 16:53

Description

Background

Use case: Export & Import process is used to synchronize data between clusters.

Data is exported from a cluster and imported into another cluster. Subsequent exports should only contain entities that have been created or updated since the last export. This helps makes the export process quicker and the payload will be smaller.

Implementation Approach

Add new fetchType: incremental, with additional parameter fromTime which is a timestamp. During Export process, entities' that have __modificationTimestamp greater than fromTime would be included in the export. 

Issue Links

• links to: Code review