[ATLAS-3663] Import Service: Migration Import: Provide Ability to Resume from Last Successfully Imported State

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: trunk
• Fix Version/s: trunk
• Component/s: atlas-core
• Assignee: Ashutosh Mestry
• Reporter: Ashutosh Mestry
• Due: 13/Mar/20
• Created: 13/Mar/20 00:07
• Updated: 16/Mar/20 21:24
• Resolved: 16/Mar/20 21:24

Description

Background

The new Migration Import implementation allows for importing large zip files. These imports can potentially run for long time. If the server shuts down while an import is in progress, there needs to be a mechanism so that the import resumes from the last imported entity.

Solution

• Implement DataMigrationStatusService this will maintain a special vertex one of whose property will be the last successfully imported status.
• Update this vertex on each successfully recorded import.
• At the start of ZipFileBaseMigration check for existence of this vertex. If it exists and has a valid position, use this to set the startPosition in the AtlasImportRequest. 

Issue Links

• links to: Code Review