[ATLAS-2460] Data Migration: Moving Data from Earlier Versions of Atlas to Most Recent

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.0
• Fix Version/s: 1.0.0
• Component/s: atlas-core
• Assignee: Ashutosh Mestry
• Reporter: Ashutosh Mestry
• Due: 28/Feb/18
• Created: 23/Feb/18 21:49
• Updated: 17/May/18 01:13
• Resolved: 11/Apr/18 20:54

Description

Background

The most recent version of Atlas uses JanusGraph as database. Earlier versions used to use Titan v0.5.4. The formats used for storing data have changed.

The current version of Atlas also implement features within Atlas entities. This makes the storages structures differ.

A data migration approach thus becomes necessary to address the format incompatibilities.

Approach

Earlier version of Atlas could use Export process to extract data out of the Titan database. This ZIP file would then be moved to a cluster with newer version of Atlas installed. An Import process on the new cluster would update the new cluster with data migrated to the new format.

It should be possible for Data Migration to be initiated from Ambari, so that it becomes part of the Ambari's upgrade process.

It should also be possible to see the status of the migration as it progresses. During the process, no hook messages should be processed or any notifications sent out.