Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-2011

Eliminate duplicate join in Pregel

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: GraphX
    • Labels:
      None

      Description

      In the iteration loop, Pregel currently performs an innerJoin to apply messages to vertices followed by an outerJoinVertices to join the resulting subset of vertices back to the graph. These two operations could be merged into a single call to joinVertices, which should be reimplemented in a more efficient manner. This would allow us to examine only the vertices that received messages.

        Attachments

          Activity

            People

            • Assignee:
              ankurd Ankur Dave
              Reporter:
              ankurd Ankur Dave
            • Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: