Uploaded image for project: 'TinkerPop'
  1. TinkerPop
  2. TINKERPOP-1193

Add a LocalBarrier interface.

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Implemented
    • 3.1.1-incubating
    • 3.2.0-incubating
    • process
    • None

    Description

      Steps like NoOpCollectingBarrier and AggregateStep are Barrier-steps, but they do not need to reduce their results down to a single machine in OLAP. In fact, it is wildly expensive to do so. We should have a LocalBarrier interface which allows such steps to "barrier" but not reduce. What happens is the Steps will hold the data on the workers, merge "nothing" to master. Then master will respawn "nothing" back to the workers and the steps continue their execution. In this way, we know the barriers maintain their synchronization, but there is not unnecessary data shuffling around the cluster.

          • Note that SupplyBarrierStep is analogous to this where its "nothing" reduction is simply a single boolean that gets and'd but means nothing.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            okram Marko A. Rodriguez
            okram Marko A. Rodriguez
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment