Uploaded image for project: 'Giraph (Retired)'
  1. Giraph (Retired)
  2. GIRAPH-314

Implement better message grouping to improve performance in SimpleTriangleClosingVertex

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Trivial
    • Resolution: Won't Fix
    • 1.0.0
    • 1.0.0
    • examples
    • None

    Description

      After running SimpleTriangleClosingVertex at scale I'm thinking the sendMessageToAllEdges() is pretty in the code, but its not a good idea in practice since each vertex V sends degree(V)^2 messages right in the first superset in this algorithm. Could do something with a combiner etc. but just grouping messages by hand at the application level by using IntArrayListWritable again does the trick fine.

      Probably should have just done it this way before, but sendMessageToAllEdges() looked so nice. Sigh. Changed unit tests to reflect this new approach, passes mvn verify and cluster, etc.

      Attachments

        1. GIRAPH-314-4.patch
          14 kB
          Eli Reisman
        2. GIRAPH-314-3.patch
          14 kB
          Eli Reisman
        3. GIRAPH-314-2.patch
          12 kB
          Eli Reisman
        4. GIRAPH-314-1.patch
          7 kB
          Eli Reisman

        Issue Links

          relates to

          Improvement - An improvement or enhancement to an existing feature or task. GIRAPH-322 Run Length Encoding for Vertex#sendMessageToAllEdges might curb out of control message growth in large scale jobs

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            People

              initialcontext Eli Reisman
              initialcontext Eli Reisman
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: