Uploaded image for project: 'TinkerPop'
  1. TinkerPop
  2. TINKERPOP-728

Improve Remote Graph Object Treatment in Console

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Implemented
    • Affects Version/s: 3.0.0-incubating
    • Fix Version/s: 3.1.0-incubating
    • Component/s: console, io, tinkergraph
    • Labels:
      None

      Description

      The current method of moving an entire Graph from server to console over :remote looks like this:

      gremlin> :import org.apache.tinkerpop.gremlin.structure.util.star.*
      ==>groovy.grape.Grape, org.apache.commons.configuration.*, .... , org.apache.tinkerpop.gremlin.structure.util.star.*
      gremlin> :remote connect tinkerpop.server conf/remote-objects.yaml
      ==>Connected - localhost/127.0.0.1:8182
      gremlin> :> g.V().map{StarGraph.of(it.get())}  // assume this is the "subgraph" - the key is to convert Vertex to StarGraph
      ==>stargraph[starOf:v[1]]
      ==>stargraph[starOf:v[2]]
      ==>stargraph[starOf:v[3]]
      ==>stargraph[starOf:v[4]]
      ==>stargraph[starOf:v[5]]
      ==>stargraph[starOf:v[6]]
      gremlin> graph = TinkerGraph.open()
      ==>tinkergraph[vertices:0 edges:0]
      gremlin> g = graph.traversal()
      ==>graphtraversalsource[tinkergraph[vertices:0 edges:0], standard]
      gremlin> result.each{it.get(StarGraph).getStarVertex().edges(OUT).each{e->e.attach(Attachable.Method.getOrCreate(graph))}}
      ==>result{object=stargraph[starOf:v[1]] class=org.apache.tinkerpop.gremlin.structure.util.star.StarGraph}
      ==>result{object=stargraph[starOf:v[2]] class=org.apache.tinkerpop.gremlin.structure.util.star.StarGraph}
      ==>result{object=stargraph[starOf:v[3]] class=org.apache.tinkerpop.gremlin.structure.util.star.StarGraph}
      ==>result{object=stargraph[starOf:v[4]] class=org.apache.tinkerpop.gremlin.structure.util.star.StarGraph}
      ==>result{object=stargraph[starOf:v[5]] class=org.apache.tinkerpop.gremlin.structure.util.star.StarGraph}
      ==>result{object=stargraph[starOf:v[6]] class=org.apache.tinkerpop.gremlin.structure.util.star.StarGraph}
      gremlin> graph
      ==>tinkergraph[vertices:6 edges:6]
      

      Would be nice if this was a bit more streamlined.

        Activity

        Hide
        spmallette stephen mallette added a comment -

        Something to consider with the approach above that Marko A. Rodriguez noted: "when you getOrCreate edge and the vertices don't exist, it uses default "vertex" label for the vertices."

        Show
        spmallette stephen mallette added a comment - Something to consider with the approach above that Marko A. Rodriguez noted: "when you getOrCreate edge and the vertices don't exist, it uses default "vertex" label for the vertices."
        Hide
        spmallette stephen mallette added a comment -

        This ended up being low-hanging fruit. Basically serialized TinkerGraph to bytes via writeGraph in a custom gryo serializer. That enables Gremlin Server to ship it back to the Gremlin Console. This is really only useful for sending around small subgraphs. Like a user subgraphs something out to TinkerGraph on the server then want to analyze it locally.

        Still need to write a few more tests and maybe add a note or two to the docs somewhere to finish this up.

        Show
        spmallette stephen mallette added a comment - This ended up being low-hanging fruit. Basically serialized TinkerGraph to bytes via writeGraph in a custom gryo serializer. That enables Gremlin Server to ship it back to the Gremlin Console. This is really only useful for sending around small subgraphs. Like a user subgraphs something out to TinkerGraph on the server then want to analyze it locally. Still need to write a few more tests and maybe add a note or two to the docs somewhere to finish this up.
        Hide
        dmill Dylan Millikin added a comment -

        Did you ever add this to the documentation? I wouldn't mind a link (or other) to see how the current feature works.
        Thanks!

        Show
        dmill Dylan Millikin added a comment - Did you ever add this to the documentation? I wouldn't mind a link (or other) to see how the current feature works. Thanks!
        Hide
        spmallette stephen mallette added a comment -

        I thought I had wrote something in the docs when I closed this but now I don't see anything. There really isn't much to it - just return a TinkerGraph from your Gremlin and it should return it. The idea here was to make this work nicely with the Gremlin Console though - not sure it works outside of gryo. If you try it with JSON and it doesn't work I think we could make a change that would allow it to work - just open a fresh issue - we could add it to 3.0.2 if that feature is useful to you.

        Show
        spmallette stephen mallette added a comment - I thought I had wrote something in the docs when I closed this but now I don't see anything. There really isn't much to it - just return a TinkerGraph from your Gremlin and it should return it. The idea here was to make this work nicely with the Gremlin Console though - not sure it works outside of gryo. If you try it with JSON and it doesn't work I think we could make a change that would allow it to work - just open a fresh issue - we could add it to 3.0.2 if that feature is useful to you.
        Hide
        dmill Dylan Millikin added a comment -

        Thanks stephen. I'll give this a look. I tried with titan today (3.0.1 console and 1.0.0 titan) and I got a bunch of errors. I'll give it another shot with a clean install and report back.

        Show
        dmill Dylan Millikin added a comment - Thanks stephen. I'll give this a look. I tried with titan today (3.0.1 console and 1.0.0 titan) and I got a bunch of errors. I'll give it another shot with a clean install and report back.
        Hide
        spmallette stephen mallette added a comment -

        Looks like there is a problem with the approach I took to implementing this:

        https://github.com/apache/incubator-tinkerpop/blob/5a8b61c09868e22a415c0469a2737d68acce9a4e/tinkergraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerIoRegistry.java#L65

        By using the serializer to TinkerGraph to serialize the Graph we don't get external IoRegistry assignments. So if a Titan RelationIdentifier slips in it won't work.

        Show
        spmallette stephen mallette added a comment - Looks like there is a problem with the approach I took to implementing this: https://github.com/apache/incubator-tinkerpop/blob/5a8b61c09868e22a415c0469a2737d68acce9a4e/tinkergraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerIoRegistry.java#L65 By using the serializer to TinkerGraph to serialize the Graph we don't get external IoRegistry assignments. So if a Titan RelationIdentifier slips in it won't work.
        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user christianbellinaef opened a pull request:

        https://github.com/apache/incubator-tinkerpop/pull/99

        tinkergraph and titan serialization

        This was one approach we took to resolving the issue discussed in https://groups.google.com/forum/#!topic/gremlin-users/apFfQW3DkK4 and https://issues.apache.org/jira/browse/TINKERPOP3-728

        If you would like to proceed with it, let us know and i can add test coverage as well as any other input you may have

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/englishtown/incubator-tinkerpop tinkergraph-deserialization

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/incubator-tinkerpop/pull/99.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #99


        commit 0eb2362682978722e3a6eb4a92d2f3a7a5018200
        Author: Christian Bellina <christian.bellina@ef.com>
        Date: 2015-09-29T14:42:21Z

        TINKERPOP3-728 pass existing kryo instance for GryoIO to allow tinkergraph deserialization to recognize already registered classes


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user christianbellinaef opened a pull request: https://github.com/apache/incubator-tinkerpop/pull/99 tinkergraph and titan serialization This was one approach we took to resolving the issue discussed in https://groups.google.com/forum/#!topic/gremlin-users/apFfQW3DkK4 and https://issues.apache.org/jira/browse/TINKERPOP3-728 If you would like to proceed with it, let us know and i can add test coverage as well as any other input you may have You can merge this pull request into a Git repository by running: $ git pull https://github.com/englishtown/incubator-tinkerpop tinkergraph-deserialization Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-tinkerpop/pull/99.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #99 commit 0eb2362682978722e3a6eb4a92d2f3a7a5018200 Author: Christian Bellina <christian.bellina@ef.com> Date: 2015-09-29T14:42:21Z TINKERPOP3-728 pass existing kryo instance for GryoIO to allow tinkergraph deserialization to recognize already registered classes
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user christianbellinaef closed the pull request at:

        https://github.com/apache/incubator-tinkerpop/pull/99

        Show
        githubbot ASF GitHub Bot added a comment - Github user christianbellinaef closed the pull request at: https://github.com/apache/incubator-tinkerpop/pull/99
        Hide
        spmallette stephen mallette added a comment -

        Thanks to this PR: https://github.com/apache/incubator-tinkerpop/pull/100 this issue is now resolved. The same kryo instance gets used as was configured by the graph, so registered serializers end up being present.

        Show
        spmallette stephen mallette added a comment - Thanks to this PR: https://github.com/apache/incubator-tinkerpop/pull/100 this issue is now resolved. The same kryo instance gets used as was configured by the graph, so registered serializers end up being present.

          People

          • Assignee:
            spmallette stephen mallette
            Reporter:
            spmallette stephen mallette
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development