[ATLAS-3796] Migration Import Display: StatusReporter Does not Remove Processed Items

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: trunk
• Fix Version/s: trunk
• Component/s: atlas-intg
• Assignee: Ashutosh Mestry
• Reporter: Ashutosh Mestry
• Due: 14/May/20
• Created: 13/May/20 17:45
• Updated: 02/Jun/20 20:36
• Resolved: 18/May/20 04:54

Description

Background

Migration Import implementation introduced StatusReporter. The class maintains GUIDs of entities that are to be processed and entities that are processed. When the entity is marked as processed, it needs to be removed from the processed set.

Steps to Duplicate

Pre-requisites

Large migration import dataset (over 50 M entities).

Steps to Duplicate

• Start importing the dataset.
• After over 60%, the Atlas server will crash with out of memory exception.

Expected results: Migration Import should continue without problems.

Actual results: Atlas server crashes with out of memory exception, hprof dump indicates large number of items allocated by StatusReporter.processedSet.

Additional information: This bug does not affect status reporting.

Work around: Resume migration import. It will continue without problems.

Issue Links

• links to: Pre-commit