Uploaded image for project: 'TinkerPop'
  1. TinkerPop
  2. TINKERPOP-1870

n^2 synchronious operation in OLAP WorkerExecutor.execute() method

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Implemented
    • Affects Version/s: 3.2.7, 3.3.1
    • Fix Version/s: 3.2.8, 3.3.2
    • Component/s: hadoop
    • Labels:
      None

      Description

      https://github.com/apache/tinkerpop/blob/master/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/WorkerExecutor.java#L80-L93

      This block of code iterates over all remote traverses to select one related to the current vertex and remove it. This operation is repeated for the next vertex and so one. For following example query it means n^2 operations (n is number of vertices). All of them in sync block. multi core spark executor will do this operations serial. 

      g.V().emit().repeat(both().dedup()).count().next()
      

      See jvisualvm screenshot. 

        Attachments

        1. findTraverserFixed.png
          180 kB
          Artem Aliev
        2. findTraverser1.png
          209 kB
          Artem Aliev
        3. findTraverser2.png
          240 kB
          Artem Aliev

          Activity

            People

            • Assignee:
              spmallette Stephen Mallette
              Reporter:
              artem.aliev Artem Aliev
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: