[TINKERPOP-1870] n^2 synchronious operation in OLAP WorkerExecutor.execute() method

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Implemented
• Affects Version/s: 3.2.7, 3.3.1
• Fix Version/s: 3.2.8, 3.3.2
• Component/s: hadoop
• Assignee: Stephen Mallette
• Reporter: Artem Aliev
• Created: 15/Jan/18 15:20
• Updated: 03/Apr/18 17:24
• Resolved: 03/Apr/18 17:24

Description

https://github.com/apache/tinkerpop/blob/master/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/WorkerExecutor.java#L80-L93

This block of code iterates over all remote traverses to select one related to the current vertex and remove it. This operation is repeated for the next vertex and so one. For following example query it means n^2 operations (n is number of vertices). All of them in sync block. multi core spark executor will do this operations serial. 

g.V().emit().repeat(both().dedup()).count().next()

See jvisualvm screenshot. 

Issue Links

• links to: GitHub Pull Request #778
• links to: GitHub Pull Request #789