Uploaded image for project: 'TinkerPop'
  1. TinkerPop
  2. TINKERPOP-1118

SparkGraphComputer should use StarGraph, not VertexWritable.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.1.1-incubating
    • None
    • hadoop

    Description

      SparkGraphComputer input RDDs are typed as:

      JavaPairRDD<Object,VertexWritable>
      

      The VertexWritable usage is a vestige from Hadoop and Giraph. In Spark, we don't need to have this wrapper and thus, we can reduce the overhead (one less object header) by making the input RDDs typed as:

      JavaPairRDD<Object,StarGraph>
      

      This would be a breaking change for graph providers that implement their own InputRDD and OutputRDD, however, the fix is trivial. Instead of new VertexWritable(vertex), they would simply do StarGraph.of(vertex).

      Attachments

        Activity

          People

            Unassigned Unassigned
            okram Marko A. Rodriguez
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: