Uploaded image for project: 'TinkerPop'
  1. TinkerPop
  2. TINKERPOP-728

Improve Remote Graph Object Treatment in Console

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Implemented
    • Affects Version/s: 3.0.0-incubating
    • Fix Version/s: 3.1.0-incubating
    • Component/s: console, io, tinkergraph
    • Labels:
      None

      Description

      The current method of moving an entire Graph from server to console over :remote looks like this:

      gremlin> :import org.apache.tinkerpop.gremlin.structure.util.star.*
      ==>groovy.grape.Grape, org.apache.commons.configuration.*, .... , org.apache.tinkerpop.gremlin.structure.util.star.*
      gremlin> :remote connect tinkerpop.server conf/remote-objects.yaml
      ==>Connected - localhost/127.0.0.1:8182
      gremlin> :> g.V().map{StarGraph.of(it.get())}  // assume this is the "subgraph" - the key is to convert Vertex to StarGraph
      ==>stargraph[starOf:v[1]]
      ==>stargraph[starOf:v[2]]
      ==>stargraph[starOf:v[3]]
      ==>stargraph[starOf:v[4]]
      ==>stargraph[starOf:v[5]]
      ==>stargraph[starOf:v[6]]
      gremlin> graph = TinkerGraph.open()
      ==>tinkergraph[vertices:0 edges:0]
      gremlin> g = graph.traversal()
      ==>graphtraversalsource[tinkergraph[vertices:0 edges:0], standard]
      gremlin> result.each{it.get(StarGraph).getStarVertex().edges(OUT).each{e->e.attach(Attachable.Method.getOrCreate(graph))}}
      ==>result{object=stargraph[starOf:v[1]] class=org.apache.tinkerpop.gremlin.structure.util.star.StarGraph}
      ==>result{object=stargraph[starOf:v[2]] class=org.apache.tinkerpop.gremlin.structure.util.star.StarGraph}
      ==>result{object=stargraph[starOf:v[3]] class=org.apache.tinkerpop.gremlin.structure.util.star.StarGraph}
      ==>result{object=stargraph[starOf:v[4]] class=org.apache.tinkerpop.gremlin.structure.util.star.StarGraph}
      ==>result{object=stargraph[starOf:v[5]] class=org.apache.tinkerpop.gremlin.structure.util.star.StarGraph}
      ==>result{object=stargraph[starOf:v[6]] class=org.apache.tinkerpop.gremlin.structure.util.star.StarGraph}
      gremlin> graph
      ==>tinkergraph[vertices:6 edges:6]
      

      Would be nice if this was a bit more streamlined.

        Activity

        Hide
        spmallette stephen mallette added a comment -

        Thanks to this PR: https://github.com/apache/incubator-tinkerpop/pull/100 this issue is now resolved. The same kryo instance gets used as was configured by the graph, so registered serializers end up being present.

        Show
        spmallette stephen mallette added a comment - Thanks to this PR: https://github.com/apache/incubator-tinkerpop/pull/100 this issue is now resolved. The same kryo instance gets used as was configured by the graph, so registered serializers end up being present.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user christianbellinaef closed the pull request at:

        https://github.com/apache/incubator-tinkerpop/pull/99

        Show
        githubbot ASF GitHub Bot added a comment - Github user christianbellinaef closed the pull request at: https://github.com/apache/incubator-tinkerpop/pull/99
        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user christianbellinaef opened a pull request:

        https://github.com/apache/incubator-tinkerpop/pull/99

        tinkergraph and titan serialization

        This was one approach we took to resolving the issue discussed in https://groups.google.com/forum/#!topic/gremlin-users/apFfQW3DkK4 and https://issues.apache.org/jira/browse/TINKERPOP3-728

        If you would like to proceed with it, let us know and i can add test coverage as well as any other input you may have

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/englishtown/incubator-tinkerpop tinkergraph-deserialization

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/incubator-tinkerpop/pull/99.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #99


        commit 0eb2362682978722e3a6eb4a92d2f3a7a5018200
        Author: Christian Bellina <christian.bellina@ef.com>
        Date: 2015-09-29T14:42:21Z

        TINKERPOP3-728 pass existing kryo instance for GryoIO to allow tinkergraph deserialization to recognize already registered classes


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user christianbellinaef opened a pull request: https://github.com/apache/incubator-tinkerpop/pull/99 tinkergraph and titan serialization This was one approach we took to resolving the issue discussed in https://groups.google.com/forum/#!topic/gremlin-users/apFfQW3DkK4 and https://issues.apache.org/jira/browse/TINKERPOP3-728 If you would like to proceed with it, let us know and i can add test coverage as well as any other input you may have You can merge this pull request into a Git repository by running: $ git pull https://github.com/englishtown/incubator-tinkerpop tinkergraph-deserialization Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-tinkerpop/pull/99.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #99 commit 0eb2362682978722e3a6eb4a92d2f3a7a5018200 Author: Christian Bellina <christian.bellina@ef.com> Date: 2015-09-29T14:42:21Z TINKERPOP3-728 pass existing kryo instance for GryoIO to allow tinkergraph deserialization to recognize already registered classes
        Hide
        spmallette stephen mallette added a comment -

        Looks like there is a problem with the approach I took to implementing this:

        https://github.com/apache/incubator-tinkerpop/blob/5a8b61c09868e22a415c0469a2737d68acce9a4e/tinkergraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerIoRegistry.java#L65

        By using the serializer to TinkerGraph to serialize the Graph we don't get external IoRegistry assignments. So if a Titan RelationIdentifier slips in it won't work.

        Show
        spmallette stephen mallette added a comment - Looks like there is a problem with the approach I took to implementing this: https://github.com/apache/incubator-tinkerpop/blob/5a8b61c09868e22a415c0469a2737d68acce9a4e/tinkergraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerIoRegistry.java#L65 By using the serializer to TinkerGraph to serialize the Graph we don't get external IoRegistry assignments. So if a Titan RelationIdentifier slips in it won't work.
        Hide
        dmill Dylan Millikin added a comment -

        Thanks stephen. I'll give this a look. I tried with titan today (3.0.1 console and 1.0.0 titan) and I got a bunch of errors. I'll give it another shot with a clean install and report back.

        Show
        dmill Dylan Millikin added a comment - Thanks stephen. I'll give this a look. I tried with titan today (3.0.1 console and 1.0.0 titan) and I got a bunch of errors. I'll give it another shot with a clean install and report back.
        Hide
        spmallette stephen mallette added a comment -

        I thought I had wrote something in the docs when I closed this but now I don't see anything. There really isn't much to it - just return a TinkerGraph from your Gremlin and it should return it. The idea here was to make this work nicely with the Gremlin Console though - not sure it works outside of gryo. If you try it with JSON and it doesn't work I think we could make a change that would allow it to work - just open a fresh issue - we could add it to 3.0.2 if that feature is useful to you.

        Show
        spmallette stephen mallette added a comment - I thought I had wrote something in the docs when I closed this but now I don't see anything. There really isn't much to it - just return a TinkerGraph from your Gremlin and it should return it. The idea here was to make this work nicely with the Gremlin Console though - not sure it works outside of gryo. If you try it with JSON and it doesn't work I think we could make a change that would allow it to work - just open a fresh issue - we could add it to 3.0.2 if that feature is useful to you.
        Hide
        dmill Dylan Millikin added a comment -

        Did you ever add this to the documentation? I wouldn't mind a link (or other) to see how the current feature works.
        Thanks!

        Show
        dmill Dylan Millikin added a comment - Did you ever add this to the documentation? I wouldn't mind a link (or other) to see how the current feature works. Thanks!
        Hide
        spmallette stephen mallette added a comment -

        This ended up being low-hanging fruit. Basically serialized TinkerGraph to bytes via writeGraph in a custom gryo serializer. That enables Gremlin Server to ship it back to the Gremlin Console. This is really only useful for sending around small subgraphs. Like a user subgraphs something out to TinkerGraph on the server then want to analyze it locally.

        Still need to write a few more tests and maybe add a note or two to the docs somewhere to finish this up.

        Show
        spmallette stephen mallette added a comment - This ended up being low-hanging fruit. Basically serialized TinkerGraph to bytes via writeGraph in a custom gryo serializer. That enables Gremlin Server to ship it back to the Gremlin Console. This is really only useful for sending around small subgraphs. Like a user subgraphs something out to TinkerGraph on the server then want to analyze it locally. Still need to write a few more tests and maybe add a note or two to the docs somewhere to finish this up.
        Hide
        spmallette stephen mallette added a comment -

        Something to consider with the approach above that Marko A. Rodriguez noted: "when you getOrCreate edge and the vertices don't exist, it uses default "vertex" label for the vertices."

        Show
        spmallette stephen mallette added a comment - Something to consider with the approach above that Marko A. Rodriguez noted: "when you getOrCreate edge and the vertices don't exist, it uses default "vertex" label for the vertices."

          People

          • Assignee:
            spmallette stephen mallette
            Reporter:
            spmallette stephen mallette
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development