Uploaded image for project: 'Giraph'
  1. Giraph
  2. GIRAPH-155

Allow creation of graph by adding edges that span multiple workers

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: None
    • Component/s: graph, lib
    • Labels:
      None

      Description

      Currently a graph is created only be adding vertices. The typical way is to read input text files line-by-line with each line describing a vertex (its value, its edges etc). The current API allows for the creation of a vertex only if all the information for the vertex is available in a single line.

      However, it's common to have graphs described in the form of edges. Edges might span multiple lines in an input file or even span multiple workers. The current API doesn't allow this. In the input superstep, a vertex must be created by a single worker.

      Instead, it should be possible for multiple workers to mutate the graph during the input superstep.

      This has the following implications:
      1) Instead of just instantiating a vertex, a vertex reader should be able to do vertex addition and edge addition requests.
      2) Multiple workers might try to create the same vertex. Any conflicts should be handled with a VertexResolver. So the resolver has to be instantiated before load time.

        Attachments

        1. GIRAPH-155.patch
          207 kB
          Alessandro Presta
        2. GIRAPH-155.patch
          197 kB
          Alessandro Presta
        3. GIRAPH-155.patch
          196 kB
          Alessandro Presta
        4. GIRAPH-155.patch
          197 kB
          Alessandro Presta

          Issue Links

            Activity

              People

              • Assignee:
                apresta Alessandro Presta
                Reporter:
                dlogothetis Dionysios Logothetis
              • Votes:
                2 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: