-
Type:
Improvement
-
Status: Resolved
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: trunk
-
Component/s: atlas-core
-
Labels:None
Background
The Java patch framework (now called data patching framework) introduced recently performs patching at the rate of 1 million entities per 15 hrs. This can be improved.
Proposed Solution
- Use the Producer-Consumer framework to spawn multiple workers to perform concurrent updates to entity vertices.
- Use AtlasGraph in bulk loading mode to further gain performance.
- Perform duplicate data checks during processing.
Projected Performance Improvement
- Based on various tests, these give increased throughput. New rate can be ~300K entities per 5 mins.