Details
-
Improvement
-
Status: Patch Available
-
Major
-
Resolution: Unresolved
-
1.1.0
-
None
-
None
Description
While doing some performance tuning I discovered that loading byte array partitions was performing slower than expected. I found that the extra time was being spent allocating a new vertex object for each distinct vertexId encountered (because vertexId object is the map key). Similar to GIRAPH-704, the use of primitive maps can provide significant performance benefit in this situation. By using a primitive map, the vertex object on the VertexIterator can be reused perpetually because the vertexId object isn't used as the map key.
When processing a large graph with 4B vertices the worker vertices requests were taking ~15 seconds each, but after implementing the above suggestion that number dropped down sub-second.