The Java patch framework (now called data patching framework) introduced recently performs patching at the rate of 1 million entities per 15 hrs. This can be improved.
- Use the Producer-Consumer framework to spawn multiple workers to perform concurrent updates to entity vertices.
- Use AtlasGraph in bulk loading mode to further gain performance.
- Perform duplicate data checks during processing.
Projected Performance Improvement
- Based on various tests, these give increased throughput. New rate can be ~300K entities per 5 mins.