[ATLAS-3396] ZipSource: Usage of In-memory Maps Limits the Size of Imports Performed

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.8.4
• Fix Version/s: 0.8.4, trunk
• Component/s: atlas-core
• Assignee: Ashutosh Mestry
• Reporter: Ashutosh Mestry
• Created: 03/Sep/19 04:41
• Updated: 21/Sep/19 04:53
• Resolved: 13/Sep/19 21:06

Description

Background

ZipSource is the container for working with Atlas-exported ZIP files. The current implementation fetches the contents of the entire ZIP file in memory. This greatly limits the size of ZIP files that can be imported.

Solution Guidance

New class ZipSourceWIthBackingDirectory, this will use a user-specified (or default) temporary directory to preserve the contents of the ZIP file. This will reduce the memory burden and large ZIP files can be imported.

Minor refactoring to ImportService will be necessary to support the new variation of EntityImportStream.

 

Issue Links

• links to: Code Review