Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-15739

Expose aggregateMessagesWithActiveSet to users.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Won't Fix
    • 1.6.1
    • None
    • GraphX
    • None

    Description

      The current version of Pregel has some flaws:

      • Each iteration expands the lineage, making stages slower as it progresses.
      • It uses the deprecated mapReduceTriplets method, which makes the interface of the sendMsg function different from aggregateMessages, bringing inconsistencies to the API.
      • It enforces an initialization stage that prevents users from having a custom state at the beginning of the process.

      It would be fairly trivial to create custom versions of Pregel that would work for many other use cases if aggregateMessagesWithActiveSet were not a private method.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              adeandrade Anderson de Andrade
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: