[ATLAS-1665] Export API: Improve Generated ZIP File Using AtlasEntityWithExtInfo

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.0
• Fix Version/s: 0.8.1, 1.0.0
• Component/s: atlas-core
• Assignee: Ashutosh Mestry
• Reporter: Ashutosh Mestry
• Due: 19/Mar/17
• Created: 17/Mar/17 18:39
• Updated: 17/May/17 16:23
• Resolved: 19/Mar/17 18:16

Description

Background

Existing implementation of Export API w.r.t ZIP file generation adds 1 .json file per entity. This makes ZIP file creation inefficient. The ZIP files are 75% larger in size than what could be possible with fewer .json file entries.

Solution

The implementation uses the new v2 API AtlasEntityWithExtInfo representation instead of AtlasEntity. This format combines an entity with related entities as one. E.g. hive_table will contain all the hive_columns that it is made up of. (See example section below.)

This results in significant reduction of generated JSON files. This impacts reduction in generated ZIP file.

Issue Links

• links to: Code Review