Uploaded image for project: 'TinkerPop'
  1. TinkerPop
  2. TINKERPOP-2256

processAllStarts of AggregateStep should only be called when barrier is empty

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: 3.3.8, 3.4.3, 3.5.0
    • Component/s: process
    • Labels:
      None

      Description

      Currently the function processNextStart , hasNextBarrier, nextBarrier of AggregateStep all call AggregateStep.processAllStarts:

          protected Traverser.Admin<S> processNextStart() {
              this.processAllStarts();
              return this.barrier.remove();
          }

      Every time we get a traverser from AggregateStep, AggregateStep.processAllStarts will be called. Then processAllStarts calls this.starts.hasNext().

          @Override
          public void processAllStarts() {
              if (this.starts.hasNext()) {
                  final BulkSet<Object> bulkSet = new BulkSet<>();
                  while (this.starts.hasNext()) {
                      final Traverser.Admin<S> traverser = this.starts.next();
                      bulkSet.add(TraversalUtil.applyNullable(traverser, this.aggregateTraversal), traverser.bulk());
                      traverser.setStepId(this.getNextStep().getId()); // when barrier is reloaded, the traversers should be at the next step
                      this.barrier.add(traverser);
                  }
                  this.getTraversal().getSideEffects().add(this.sideEffectKey, bulkSet);
              }
          }
      
      

      It results in a lot of hasNext call.

      As document says "The step uses eager evaluation in that no objects continue on until all previous objects have been fully aggregated." maybe we can limit the AggregateStep.processAllStarts only be called once.

       

       

       

      We found this when we run DSL like this :

      g.V().has('name','wxy').repeat(both("knows").simplePath()).emit().times(2).aggregate("friends")

      and the plan is like this :

      GraphStep(vertex,[name.eq(wxy)]), RepeatStep([VertexStep(BOTH,[knows],vertex), PathFilterStep(simple), RepeatEndStep],until(loops(2)),emit(true)), AggregateStep(friends)
      

      Then we found thousands of calls to GraphStep(vertex,[name.eq(wxy)]).hasNext.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                spmallette Stephen Mallette
                Reporter:
                wangxiyu191 Wang Xiyu
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: