[ATLAS-1851] Import API: Ability to Specify Starting Entity for Import

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: trunk
• Fix Version/s: 0.8.1, 1.0.0
• Component/s: atlas-core
• Assignee: Ashutosh Mestry
• Reporter: Ashutosh Mestry
• Due: 02/Jun/17
• Created: 31/May/17 23:01
• Updated: 06/Jun/17 08:42
• Resolved: 06/Jun/17 08:42

Description

Background & Scenarios

Import operations have a potential to go on for several hours. It is likely that the the operation does not complete and the server crashes.

To improve on this situation, it would be beneficial to be able to specify the last successfully imported entity using either the entity guid or the position in the entity creation order. That way, the import operation could resume from a later point rather than starting from scratch.

This improvement will thus allow Import API to resume where it last stopped.

Implementation

AtlasImportRequest should support 2 additional options:

• startGuid: Entity GUID that should be used to resume import operation.
• startPosition: Starting position of the entity within the creationOrder.

Within ZipSource this option would be used to set the internal iterator to a position from which the import can start. ImportService could read do the needful before starting to process entities.

Additional Implementation Guidance

Logs generated during import operations do not have the necessary information necessary to help the consumer with information needed to use these new options.

It would be worth while to include the entity index within creationOrder and the guid of the entity that was successfully imported.