[ATLAS-3320] Import Service: Support Concurrent Ingest

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: trunk
• Fix Version/s: trunk
• Component/s: atlas-core
• Assignee: Ashutosh Mestry
• Reporter: Ashutosh Mestry
• Due: 12/Jul/19
• Created: 08/Jul/19 16:08
• Updated: 12/Mar/20 23:59
• Resolved: 12/Mar/20 23:59

Description

Background

Existing implementation of import supports single entity creation. This greatly limits the rate of ingest.

Solution Approach

• Use existing producer-consumer framework to allow for configurable consumers of entities from the input stream.
• Modify BulkImporterImpl to implement the consumer.
• Modify AtlasImportRequest to specify the batch size and number of workers.

 

Issue Links

• links to: Code Review