Uploaded image for project: 'TinkerPop'
  1. TinkerPop
  2. TINKERPOP-1153

Add ByModulating and TimesModulating interfaces.

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Implemented
    • Affects Version/s: 3.1.2-incubating
    • Fix Version/s: 3.2.0-incubating
    • Component/s: process
    • Labels:
      None

      Description

      Currently GraphTraversal is responsible for the logic for by()- and times()-modulators. For instance, if you do:

      ...by('name')
      

      It is GraphTraversal that will compile that to __.values('name').limit(1). This should not be the logic of GraphTraversal, but instead, logic within the step being modulated. Thus, I propose:

      public interface ByModulating {
      
        public void add(String string) throws UnsupportedOperationException
        public void add(Traversal traversal) throws UnsupportedOperationException
        public void add(T t) throws UnsupportedOperationException
      
      }
      

      Likewise for TimesModulating ... and any other modulators down the road.

        Activity

        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user okram opened a pull request:

        https://github.com/apache/incubator-tinkerpop/pull/230

        TINKERPOP-1153 & TINKERPOP-1154: (Summary) Multi-OLAP compilation in a single Traversal (hardened)

        https://issues.apache.org/jira/browse/TINKERPOP-1154
        https://issues.apache.org/jira/browse/TINKERPOP-1153

        There is a lot in this pull request. While creating the infrastructure to allow us to do multi-OLAP jobs in a single Traversal compilation, I ended up cleaning numerous ancient areas of the codebase that were awkward with respects to latest developments. Everything is backwards compatible though there are lots of Deprecations. However, there is a new `GraphComptuerTest` that requires that `GraphComputer` providers support "compute key revival." I had to do a small tweak to both `TinkerGraphComputer` and `SparkGraphComputer` for them to pass the test. `GiraphGraphComputer` just handled it. For users, this ticket basically provides them:

        ```
        g.V().hasLabel('person').
        pageRank(0.85).
        by('friendRank').
        by(outE('knows')).
        times(10).
        out('worksFor').order().by('friendRank').limit(10).valueMap('name','friendRank')
        ```

        CHANGELOG

        ```

        • Added `PeerPressureVertexProgramStep` and `GraphTraversal.peerPressure()`.
        • Added `PureTraversal` for handling pure and compiled versions of a `Traversal`. Useful in OLAP.
        • Added `ScriptTraversal` which allows for delayed compilation of script-based `Traversals`.
        • Simplified `VertexProgram` implementations with a `PureTraversal`-model and deprecated `ConfigurationTraversal`.
        • Simplified script-based `Traversals` via `ScriptTraversal` and deprecated `TraversalScriptFunction` and `TraversalScriptHelper`.
        • Added `ByModulating` interface which allows the `Step` to decide how a `by()`-modulation should be handled. (breaking)
        • Simplified the `by()`-modulation patterns of `OrderGlobalStep` and `OrderLocalStep`.
        • Added `GraphComputerTest.shouldSupportPreExistingComputeKeys()` to ensure existing compute keys are "revived." (breaking)
          ```

        UPDATE DOCS

        ```
        For providers that support their own `GraphComputer` implementation, note that there is a new `GraphComputerTest.shouldSupportPreExistingComputeKeys()`. When chaining OLAP jobs together, if an OLAP job requires the compute keys of a previous OLAP job, then the existing compute keys must be accessible. A simple 2 line change to `SparkGraphComputer` and `TinkerGraphComputer` solved this for TinkerPop. `GiraphGraphComputer` did not need an update as this feature was already naturally supported.

        If the provider has custom steps that leverage `by()`-modulation, those still will now need to implement `ByModulating`. Most of the methods in `ByModulating` are `default` and, for most situations, only `ByModulating.modulateBy(Traversal)` should be implemented. Note that this method's body will most like be identical to `TraversalParent.addLocalChild()`.

        Providers that have custom Gremlin language implementations (e.g. Gremlin-Scala), there is a new class called `ScriptTraversal` which will handle script-based processing of traversals. The entire `GroovyXXXTest`-suite was updated to use this new class. The previous `TraversalScriptHelper` class has been deprecated so immediate upgrading is not required.
        ```

        REFERENCE DOCS

            • I will do these at the time of merge.

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/apache/incubator-tinkerpop TINKERPOP-1154

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/incubator-tinkerpop/pull/230.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #230


        commit f2bec67b97a6d0b1dad1392bbdd6f51525686da8
        Author: Marko A. Rodriguez <okrammarko@gmail.com>
        Date: 2016-02-15T15:30:46Z

        Added ScriptTraversal which contains a String script and reference to a traversal source and script engine. This greatly simplifies serializing traversals and will enable us to just talk always in terms of Traversal and not worry about if its a serializable or non-serializable traversal.

        commit 44f78159496737b8c0d7d714fe8dc747bfa5842a
        Author: Marko A. Rodriguez <okrammarko@gmail.com>
        Date: 2016-02-15T15:31:49Z

        revived a deprecated method for backwards compatibility.

        commit 94ffac91b39340b7ae7f2c3609c8715339183f7d
        Author: Marko A. Rodriguez <okrammarko@gmail.com>
        Date: 2016-02-15T18:25:01Z

        Added ByModulating interface which has numerous modulateBy() overloads. Interestingly, all the default call the respective Traversal-form. This makes it much easier for steps to reason about incoming traversals from by()-modulation and do something different with it. GraphTraversal no longer reasons about by(). It simply calls ((ByModulator)getEndStep()).modulateBy(...). Clean logic. Added more test cases to OrderTest as we didn't have tests for a few of the by() modulators. I believe we will be able to get rid of TraversalParent.addXXXChild()/removeXXXChild() as these were provided so that Traversal could add modulation. TINKERPOP-1153

        commit 55e5559d0ea68843a2dff4d0c3f286a6a6f29f12
        Author: Marko A. Rodriguez <okrammarko@gmail.com>
        Date: 2016-02-15T19:29:56Z

        Fixed TINKERPOP-1105. DetachedVertexProperties were being serialized as DetachedProperty by GryoSerializers.

        commit 45b097552710a4884af89bcd67868b689a918a9f
        Author: Marko A. Rodriguez <okrammarko@gmail.com>
        Date: 2016-02-15T19:48:04Z

        Merge branch 'master' into TINKERPOP-1154

        commit c3eb653cbd3cb38e0925f0c1c21f4e5761637b79
        Author: Marko A. Rodriguez <okrammarko@gmail.com>
        Date: 2016-02-16T16:31:22Z

        Simplified the configuration internals of TraversalVertexProgram, PageRankVertexProgram, and PeerPressureVertexProgram. Stubbed the ability to allow these vertex programs determine which source vertices to use for their computation. Added another PageRankTest which verifies the new ByModulating behavior for PageRankVertexProgramStep – g.V().pageRank().by('friendRank').by(outE('knows')). Introduced PureTraversal which contains an uncompiled and compiled form of a Traversal. Useful for OLAP based VertexPrograms and Steps.

        commit d06329eef60bf01da072f107b411d8554ec79203
        Author: Marko A. Rodriguez <okrammarko@gmail.com>
        Date: 2016-02-16T16:42:57Z

        added ColumnTraversal for Order-based column selection. Added more test cases to LambdaRestrictionStrategyTest. Split the test suite into 'allowed' and 'not allowed' traversals so its easier to test the scope of what is considered 'lambda'.

        commit 010c9f343df16d09266560631471147edb4f442f
        Author: Marko A. Rodriguez <okrammarko@gmail.com>
        Date: 2016-02-16T18:56:37Z

        little more work. saving what I have for a big push next.

        commit 88f24c7f5a889520655fb645010eead3e2815078
        Author: Marko A. Rodriguez <okrammarko@gmail.com>
        Date: 2016-02-16T22:15:50Z

        Through hell and back. You can now go TraversalVertexProgram->PageRankVertexProgramStep->TraversalVertexPrograph->OLTP. I had to create the concept of 'reving and old view.' VertexProgramStrategy is much more flexible. Still needs alot of work, but we are getting there.

        commit 01b4a539b9a5a2cd28ac66adcc2bf5843d5e1769
        Author: Marko A. Rodriguez <okrammarko@gmail.com>
        Date: 2016-02-16T23:17:09Z

        added GraphComputerTest.shouldSupportPreExistingComputeKeys(). This is necessary when job chaining and HALTED_TRAVERSERS are later required by another TraversalVertexProgram.

        commit 86284f2c8f6c13782e5165c37e35c7a56a66192a
        Author: Marko A. Rodriguez <okrammarko@gmail.com>
        Date: 2016-02-17T03:45:15Z

        TraversalVertexProgram now uses the GraphStep instead of reasoning on its internal state. This simplifies the code and stages us for mid-traversal V()/E(). Added a new GraphTest. Fixed an old problem with TraversalVertexProgram and headless-traversals. I just changed and old GraphComputerTest and it worked. Sweet.

        commit 969472f065f8d4a6792a512f9c3bbcc9d2c3ceb6
        Author: Marko A. Rodriguez <okrammarko@gmail.com>
        Date: 2016-02-17T12:43:40Z

        GraphTraversalSource, for all its withXXX-methods, simply calls TraversalSource.super.withXXX. Now that we have TraversalSource.getStrategies(), its super easy.

        commit 11238b630bfc9af71f0a443eada36381c9f5fa15
        Author: Marko A. Rodriguez <okrammarko@gmail.com>
        Date: 2016-02-17T16:01:47Z

        Created VertexProgramStep which is abstract and provides processNextStarts() functionality to TraversalVertexProgramStep and PageRankVertexProgramStep. Moved the PureTraversal model into the VertexProgramSteps. Added TimesModulating which allows steps to do what they wish with times(int). pageRank().times(20) for example. General cleanup.

        commit 2d5cd1c0d8ec3f687f237cdf9be4b0b74a73af56
        Author: Marko A. Rodriguez <okrammarko@gmail.com>
        Date: 2016-02-17T17:04:22Z

        VertexProgramStrategy is now smart about GraphStep.PageRank. It will just propagate the GraphStep forward to reduce the number of OLAP jobs required. Added VertexProgramStrategyTest which verifies the compilation of a OLAP traversal.

        commit 622f84b9dc94ae5ef1b6c174fcf360cef2366d37
        Author: Marko A. Rodriguez <okrammarko@gmail.com>
        Date: 2016-02-17T18:04:27Z

        Added PeerPressureVertexProgramStep and GraphTraversal.peerPressure(). Added GraphTraversal.pageRank(alpha). Ensured that VertexComputing steps are NOT used in StandardTraversal.


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user okram opened a pull request: https://github.com/apache/incubator-tinkerpop/pull/230 TINKERPOP-1153 & TINKERPOP-1154 : (Summary) Multi-OLAP compilation in a single Traversal (hardened) https://issues.apache.org/jira/browse/TINKERPOP-1154 https://issues.apache.org/jira/browse/TINKERPOP-1153 There is a lot in this pull request. While creating the infrastructure to allow us to do multi-OLAP jobs in a single Traversal compilation, I ended up cleaning numerous ancient areas of the codebase that were awkward with respects to latest developments. Everything is backwards compatible though there are lots of Deprecations. However, there is a new `GraphComptuerTest` that requires that `GraphComputer` providers support "compute key revival." I had to do a small tweak to both `TinkerGraphComputer` and `SparkGraphComputer` for them to pass the test. `GiraphGraphComputer` just handled it. For users, this ticket basically provides them: ``` g.V().hasLabel('person'). pageRank(0.85). by('friendRank'). by(outE('knows')). times(10). out('worksFor').order().by('friendRank').limit(10).valueMap('name','friendRank') ``` CHANGELOG ``` Added `PeerPressureVertexProgramStep` and `GraphTraversal.peerPressure()`. Added `PureTraversal` for handling pure and compiled versions of a `Traversal`. Useful in OLAP. Added `ScriptTraversal` which allows for delayed compilation of script-based `Traversals`. Simplified `VertexProgram` implementations with a `PureTraversal`-model and deprecated `ConfigurationTraversal`. Simplified script-based `Traversals` via `ScriptTraversal` and deprecated `TraversalScriptFunction` and `TraversalScriptHelper`. Added `ByModulating` interface which allows the `Step` to decide how a `by()`-modulation should be handled. ( breaking ) Simplified the `by()`-modulation patterns of `OrderGlobalStep` and `OrderLocalStep`. Added `GraphComputerTest.shouldSupportPreExistingComputeKeys()` to ensure existing compute keys are "revived." ( breaking ) ``` UPDATE DOCS ``` For providers that support their own `GraphComputer` implementation, note that there is a new `GraphComputerTest.shouldSupportPreExistingComputeKeys()`. When chaining OLAP jobs together, if an OLAP job requires the compute keys of a previous OLAP job, then the existing compute keys must be accessible. A simple 2 line change to `SparkGraphComputer` and `TinkerGraphComputer` solved this for TinkerPop. `GiraphGraphComputer` did not need an update as this feature was already naturally supported. If the provider has custom steps that leverage `by()`-modulation, those still will now need to implement `ByModulating`. Most of the methods in `ByModulating` are `default` and, for most situations, only `ByModulating.modulateBy(Traversal)` should be implemented. Note that this method's body will most like be identical to `TraversalParent.addLocalChild()`. Providers that have custom Gremlin language implementations (e.g. Gremlin-Scala), there is a new class called `ScriptTraversal` which will handle script-based processing of traversals. The entire `GroovyXXXTest`-suite was updated to use this new class. The previous `TraversalScriptHelper` class has been deprecated so immediate upgrading is not required. ``` REFERENCE DOCS I will do these at the time of merge. You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/incubator-tinkerpop TINKERPOP-1154 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-tinkerpop/pull/230.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #230 commit f2bec67b97a6d0b1dad1392bbdd6f51525686da8 Author: Marko A. Rodriguez <okrammarko@gmail.com> Date: 2016-02-15T15:30:46Z Added ScriptTraversal which contains a String script and reference to a traversal source and script engine. This greatly simplifies serializing traversals and will enable us to just talk always in terms of Traversal and not worry about if its a serializable or non-serializable traversal. commit 44f78159496737b8c0d7d714fe8dc747bfa5842a Author: Marko A. Rodriguez <okrammarko@gmail.com> Date: 2016-02-15T15:31:49Z revived a deprecated method for backwards compatibility. commit 94ffac91b39340b7ae7f2c3609c8715339183f7d Author: Marko A. Rodriguez <okrammarko@gmail.com> Date: 2016-02-15T18:25:01Z Added ByModulating interface which has numerous modulateBy() overloads. Interestingly, all the default call the respective Traversal-form. This makes it much easier for steps to reason about incoming traversals from by()-modulation and do something different with it. GraphTraversal no longer reasons about by(). It simply calls ((ByModulator)getEndStep()).modulateBy(...). Clean logic. Added more test cases to OrderTest as we didn't have tests for a few of the by() modulators. I believe we will be able to get rid of TraversalParent.addXXXChild()/removeXXXChild() as these were provided so that Traversal could add modulation. TINKERPOP-1153 commit 55e5559d0ea68843a2dff4d0c3f286a6a6f29f12 Author: Marko A. Rodriguez <okrammarko@gmail.com> Date: 2016-02-15T19:29:56Z Fixed TINKERPOP-1105 . DetachedVertexProperties were being serialized as DetachedProperty by GryoSerializers. commit 45b097552710a4884af89bcd67868b689a918a9f Author: Marko A. Rodriguez <okrammarko@gmail.com> Date: 2016-02-15T19:48:04Z Merge branch 'master' into TINKERPOP-1154 commit c3eb653cbd3cb38e0925f0c1c21f4e5761637b79 Author: Marko A. Rodriguez <okrammarko@gmail.com> Date: 2016-02-16T16:31:22Z Simplified the configuration internals of TraversalVertexProgram, PageRankVertexProgram, and PeerPressureVertexProgram. Stubbed the ability to allow these vertex programs determine which source vertices to use for their computation. Added another PageRankTest which verifies the new ByModulating behavior for PageRankVertexProgramStep – g.V().pageRank().by('friendRank').by(outE('knows')). Introduced PureTraversal which contains an uncompiled and compiled form of a Traversal. Useful for OLAP based VertexPrograms and Steps. commit d06329eef60bf01da072f107b411d8554ec79203 Author: Marko A. Rodriguez <okrammarko@gmail.com> Date: 2016-02-16T16:42:57Z added ColumnTraversal for Order-based column selection. Added more test cases to LambdaRestrictionStrategyTest. Split the test suite into 'allowed' and 'not allowed' traversals so its easier to test the scope of what is considered 'lambda'. commit 010c9f343df16d09266560631471147edb4f442f Author: Marko A. Rodriguez <okrammarko@gmail.com> Date: 2016-02-16T18:56:37Z little more work. saving what I have for a big push next. commit 88f24c7f5a889520655fb645010eead3e2815078 Author: Marko A. Rodriguez <okrammarko@gmail.com> Date: 2016-02-16T22:15:50Z Through hell and back. You can now go TraversalVertexProgram->PageRankVertexProgramStep->TraversalVertexPrograph->OLTP. I had to create the concept of 'reving and old view.' VertexProgramStrategy is much more flexible. Still needs alot of work, but we are getting there. commit 01b4a539b9a5a2cd28ac66adcc2bf5843d5e1769 Author: Marko A. Rodriguez <okrammarko@gmail.com> Date: 2016-02-16T23:17:09Z added GraphComputerTest.shouldSupportPreExistingComputeKeys(). This is necessary when job chaining and HALTED_TRAVERSERS are later required by another TraversalVertexProgram. commit 86284f2c8f6c13782e5165c37e35c7a56a66192a Author: Marko A. Rodriguez <okrammarko@gmail.com> Date: 2016-02-17T03:45:15Z TraversalVertexProgram now uses the GraphStep instead of reasoning on its internal state. This simplifies the code and stages us for mid-traversal V()/E(). Added a new GraphTest. Fixed an old problem with TraversalVertexProgram and headless-traversals. I just changed and old GraphComputerTest and it worked. Sweet. commit 969472f065f8d4a6792a512f9c3bbcc9d2c3ceb6 Author: Marko A. Rodriguez <okrammarko@gmail.com> Date: 2016-02-17T12:43:40Z GraphTraversalSource, for all its withXXX-methods, simply calls TraversalSource.super.withXXX. Now that we have TraversalSource.getStrategies(), its super easy. commit 11238b630bfc9af71f0a443eada36381c9f5fa15 Author: Marko A. Rodriguez <okrammarko@gmail.com> Date: 2016-02-17T16:01:47Z Created VertexProgramStep which is abstract and provides processNextStarts() functionality to TraversalVertexProgramStep and PageRankVertexProgramStep. Moved the PureTraversal model into the VertexProgramSteps. Added TimesModulating which allows steps to do what they wish with times(int). pageRank().times(20) for example. General cleanup. commit 2d5cd1c0d8ec3f687f237cdf9be4b0b74a73af56 Author: Marko A. Rodriguez <okrammarko@gmail.com> Date: 2016-02-17T17:04:22Z VertexProgramStrategy is now smart about GraphStep.PageRank. It will just propagate the GraphStep forward to reduce the number of OLAP jobs required. Added VertexProgramStrategyTest which verifies the compilation of a OLAP traversal. commit 622f84b9dc94ae5ef1b6c174fcf360cef2366d37 Author: Marko A. Rodriguez <okrammarko@gmail.com> Date: 2016-02-17T18:04:27Z Added PeerPressureVertexProgramStep and GraphTraversal.peerPressure(). Added GraphTraversal.pageRank(alpha). Ensured that VertexComputing steps are NOT used in StandardTraversal.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user okram commented on the pull request:

        https://github.com/apache/incubator-tinkerpop/pull/230#issuecomment-185429165

        I decided to write the docs. Docs written. Note that they don't build. This has nothing to do with this PR. It has to do with the recent fix to the `manifest` for Neo4j plugin. Neo4j pulls in Scala 2.11.x while Spark uses 2.10.x. The doc builder (having all plugins installed at once) makes it so Spark uses Scala 2.11.x and there are "NoMethodDefinedExceptions." This effects `tp31` branch, but not `master/` as master will (in a later ticket) bump to Spark 1.6.0 which uses Scala 2.11.x.

        Show
        githubbot ASF GitHub Bot added a comment - Github user okram commented on the pull request: https://github.com/apache/incubator-tinkerpop/pull/230#issuecomment-185429165 I decided to write the docs. Docs written. Note that they don't build. This has nothing to do with this PR. It has to do with the recent fix to the `manifest` for Neo4j plugin. Neo4j pulls in Scala 2.11.x while Spark uses 2.10.x. The doc builder (having all plugins installed at once) makes it so Spark uses Scala 2.11.x and there are "NoMethodDefinedExceptions." This effects `tp31` branch, but not `master/` as master will (in a later ticket) bump to Spark 1.6.0 which uses Scala 2.11.x.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user okram commented on the pull request:

        https://github.com/apache/incubator-tinkerpop/pull/230#issuecomment-185521157

        VOTE +1.

        ```
        [INFO] ------------------------------------------------------------------------
        [INFO] Reactor Summary:
        [INFO]
        [INFO] Apache TinkerPop .................................. SUCCESS [5.044s]
        [INFO] Apache TinkerPop :: Gremlin Shaded ................ SUCCESS [2.132s]
        [INFO] Apache TinkerPop :: Gremlin Core .................. SUCCESS [38.711s]
        [INFO] Apache TinkerPop :: Gremlin Test .................. SUCCESS [12.019s]
        [INFO] Apache TinkerPop :: Gremlin Groovy ................ SUCCESS [32.419s]
        [INFO] Apache TinkerPop :: Gremlin Groovy Test ........... SUCCESS [6.354s]
        [INFO] Apache TinkerPop :: TinkerGraph Gremlin ........... SUCCESS [3:27.877s]
        [INFO] Apache TinkerPop :: Hadoop Gremlin ................ SUCCESS [5:07.110s]
        [INFO] Apache TinkerPop :: Spark Gremlin ................. SUCCESS [10:39.819s]
        [INFO] Apache TinkerPop :: Giraph Gremlin ................ SUCCESS [2:21:44.149s]
        [INFO] Apache TinkerPop :: Neo4j Gremlin ................. SUCCESS [17:29.014s]
        [INFO] Apache TinkerPop :: Gremlin Driver ................ SUCCESS [8.910s]
        [INFO] Apache TinkerPop :: Gremlin Server ................ SUCCESS [10:56.703s]
        [INFO] Apache TinkerPop :: Gremlin Console ............... SUCCESS [1:08.703s]
        [INFO] ------------------------------------------------------------------------
        [INFO] BUILD SUCCESS
        [INFO] ------------------------------------------------------------------------
        [INFO] Total time: 3:12:19.462s
        [INFO] Finished at: Wed Feb 17 19:59:19 MST 2016
        [INFO] Final Memory: 93M/731M
        [INFO] ------------------------------------------------------------------------
        ~/software/tinkerpop/tinkerpop3$
        ```

        Show
        githubbot ASF GitHub Bot added a comment - Github user okram commented on the pull request: https://github.com/apache/incubator-tinkerpop/pull/230#issuecomment-185521157 VOTE +1. ``` [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Apache TinkerPop .................................. SUCCESS [5.044s] [INFO] Apache TinkerPop :: Gremlin Shaded ................ SUCCESS [2.132s] [INFO] Apache TinkerPop :: Gremlin Core .................. SUCCESS [38.711s] [INFO] Apache TinkerPop :: Gremlin Test .................. SUCCESS [12.019s] [INFO] Apache TinkerPop :: Gremlin Groovy ................ SUCCESS [32.419s] [INFO] Apache TinkerPop :: Gremlin Groovy Test ........... SUCCESS [6.354s] [INFO] Apache TinkerPop :: TinkerGraph Gremlin ........... SUCCESS [3:27.877s] [INFO] Apache TinkerPop :: Hadoop Gremlin ................ SUCCESS [5:07.110s] [INFO] Apache TinkerPop :: Spark Gremlin ................. SUCCESS [10:39.819s] [INFO] Apache TinkerPop :: Giraph Gremlin ................ SUCCESS [2:21:44.149s] [INFO] Apache TinkerPop :: Neo4j Gremlin ................. SUCCESS [17:29.014s] [INFO] Apache TinkerPop :: Gremlin Driver ................ SUCCESS [8.910s] [INFO] Apache TinkerPop :: Gremlin Server ................ SUCCESS [10:56.703s] [INFO] Apache TinkerPop :: Gremlin Console ............... SUCCESS [1:08.703s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 3:12:19.462s [INFO] Finished at: Wed Feb 17 19:59:19 MST 2016 [INFO] Final Memory: 93M/731M [INFO] ------------------------------------------------------------------------ ~/software/tinkerpop/tinkerpop3$ ```
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user dkuppitz commented on the pull request:

        https://github.com/apache/incubator-tinkerpop/pull/230#issuecomment-185987704

        ```
        Failed tests:
        PageRankTest$Traversals>PageRankTest.g_V_hasLabelXsoftwareX_hasXname_rippleX_pageRankX1X_byXinEXcreatedXX_timesX1X_byXpriorsX_inXcreatedX_unionXboth__identityX_valueMapXname_priorsX:176 expected:<1.0> but was:<0.0>
        ```

        Show
        githubbot ASF GitHub Bot added a comment - Github user dkuppitz commented on the pull request: https://github.com/apache/incubator-tinkerpop/pull/230#issuecomment-185987704 ``` Failed tests: PageRankTest$Traversals>PageRankTest.g_V_hasLabelXsoftwareX_hasXname_rippleX_pageRankX1X_byXinEXcreatedXX_timesX1X_byXpriorsX_inXcreatedX_unionXboth__identityX_valueMapXname_priorsX:176 expected:<1.0> but was:<0.0> ```
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user okram commented on the pull request:

        https://github.com/apache/incubator-tinkerpop/pull/230#issuecomment-186004389

        @dkuppitz ... that is weird. It all finished for me just now. Can you run the query in the console and tell me what you get:

        ```
        g.V().hasLabel("software").has("name", "ripple").pageRank(1.0).by(_.inE("created")).times(1).by("priors").in("created").union(_.both(), __.identity()).valueMap("name", "priors")
        ```

        ```
        [INFO] ------------------------------------------------------------------------
        [INFO] Reactor Summary:
        [INFO]
        [INFO] Apache TinkerPop .................................. SUCCESS [4.355s]
        [INFO] Apache TinkerPop :: Gremlin Shaded ................ SUCCESS [3.344s]
        [INFO] Apache TinkerPop :: Gremlin Core .................. SUCCESS [38.227s]
        [INFO] Apache TinkerPop :: Gremlin Test .................. SUCCESS [11.285s]
        [INFO] Apache TinkerPop :: Gremlin Groovy ................ SUCCESS [32.956s]
        [INFO] Apache TinkerPop :: Gremlin Groovy Test ........... SUCCESS [6.191s]
        [INFO] Apache TinkerPop :: TinkerGraph Gremlin ........... SUCCESS [3:16.894s]
        [INFO] Apache TinkerPop :: Hadoop Gremlin ................ SUCCESS [5:01.567s]
        [INFO] Apache TinkerPop :: Spark Gremlin ................. SUCCESS [10:15.725s]
        [INFO] Apache TinkerPop :: Giraph Gremlin ................ SUCCESS [2:24:31.663s]
        [INFO] Apache TinkerPop :: Neo4j Gremlin ................. SUCCESS [17:33.444s]
        [INFO] Apache TinkerPop :: Gremlin Driver ................ SUCCESS [9.075s]
        [INFO] Apache TinkerPop :: Gremlin Server ................ SUCCESS [10:55.872s]
        [INFO] Apache TinkerPop :: Gremlin Console ............... SUCCESS [1:08.465s]
        [INFO] ------------------------------------------------------------------------
        [INFO] BUILD SUCCESS
        [INFO] ------------------------------------------------------------------------
        [INFO] Total time: 3:14:29.536s
        [INFO] Finished at: Thu Feb 18 17:46:39 MST 2016
        [INFO] Final Memory: 95M/707M
        [INFO] ------------------------------------------------------------------------
        ```

        Show
        githubbot ASF GitHub Bot added a comment - Github user okram commented on the pull request: https://github.com/apache/incubator-tinkerpop/pull/230#issuecomment-186004389 @dkuppitz ... that is weird. It all finished for me just now. Can you run the query in the console and tell me what you get: ``` g.V().hasLabel("software").has("name", "ripple").pageRank(1.0).by(_ .inE("created")).times(1).by("priors").in("created").union( _.both(), __.identity()).valueMap("name", "priors") ``` ``` [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Apache TinkerPop .................................. SUCCESS [4.355s] [INFO] Apache TinkerPop :: Gremlin Shaded ................ SUCCESS [3.344s] [INFO] Apache TinkerPop :: Gremlin Core .................. SUCCESS [38.227s] [INFO] Apache TinkerPop :: Gremlin Test .................. SUCCESS [11.285s] [INFO] Apache TinkerPop :: Gremlin Groovy ................ SUCCESS [32.956s] [INFO] Apache TinkerPop :: Gremlin Groovy Test ........... SUCCESS [6.191s] [INFO] Apache TinkerPop :: TinkerGraph Gremlin ........... SUCCESS [3:16.894s] [INFO] Apache TinkerPop :: Hadoop Gremlin ................ SUCCESS [5:01.567s] [INFO] Apache TinkerPop :: Spark Gremlin ................. SUCCESS [10:15.725s] [INFO] Apache TinkerPop :: Giraph Gremlin ................ SUCCESS [2:24:31.663s] [INFO] Apache TinkerPop :: Neo4j Gremlin ................. SUCCESS [17:33.444s] [INFO] Apache TinkerPop :: Gremlin Driver ................ SUCCESS [9.075s] [INFO] Apache TinkerPop :: Gremlin Server ................ SUCCESS [10:55.872s] [INFO] Apache TinkerPop :: Gremlin Console ............... SUCCESS [1:08.465s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 3:14:29.536s [INFO] Finished at: Thu Feb 18 17:46:39 MST 2016 [INFO] Final Memory: 95M/707M [INFO] ------------------------------------------------------------------------ ```
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user spmallette commented on the pull request:

        https://github.com/apache/incubator-tinkerpop/pull/230#issuecomment-186209068

        looks like travis shows the same error as what @dkuppitz got

        Show
        githubbot ASF GitHub Bot added a comment - Github user spmallette commented on the pull request: https://github.com/apache/incubator-tinkerpop/pull/230#issuecomment-186209068 looks like travis shows the same error as what @dkuppitz got
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user okram commented on the pull request:

        https://github.com/apache/incubator-tinkerpop/pull/230#issuecomment-186256034

        So weird. Again, overnight.

        ```
        [INFO] ------------------------------------------------------------------------
        [INFO] Reactor Summary:
        [INFO]
        [INFO] Apache TinkerPop .................................. SUCCESS [5.222s]
        [INFO] Apache TinkerPop :: Gremlin Shaded ................ SUCCESS [2.530s]
        [INFO] Apache TinkerPop :: Gremlin Core .................. SUCCESS [38.401s]
        [INFO] Apache TinkerPop :: Gremlin Test .................. SUCCESS [11.325s]
        [INFO] Apache TinkerPop :: Gremlin Groovy ................ SUCCESS [32.358s]
        [INFO] Apache TinkerPop :: Gremlin Groovy Test ........... SUCCESS [6.013s]
        [INFO] Apache TinkerPop :: TinkerGraph Gremlin ........... SUCCESS [3:15.039s]
        [INFO] Apache TinkerPop :: Hadoop Gremlin ................ SUCCESS [5:03.753s]
        [INFO] Apache TinkerPop :: Spark Gremlin ................. SUCCESS [10:01.122s]
        [INFO] Apache TinkerPop :: Giraph Gremlin ................ SUCCESS [2:24:33.823s]
        [INFO] Apache TinkerPop :: Neo4j Gremlin ................. SUCCESS [17:40.001s]
        [INFO] Apache TinkerPop :: Gremlin Driver ................ SUCCESS [8.919s]
        [INFO] Apache TinkerPop :: Gremlin Server ................ SUCCESS [10:56.373s]
        [INFO] Apache TinkerPop :: Gremlin Console ............... SUCCESS [1:08.209s]
        [INFO] ------------------------------------------------------------------------
        [INFO] BUILD SUCCESS
        [INFO] ------------------------------------------------------------------------
        [INFO] Total time: 3:14:23.560s
        [INFO] Finished at: Thu Feb 18 21:43:01 MST 2016
        [INFO] Final Memory: 94M/708M
        [INFO] ------------------------------------------------------------------------
        ```

        Here I will just comment out that test. If you guys can tell me what result of the query is in the console I can fix it later, but without it failing here, I have no idea.

        Show
        githubbot ASF GitHub Bot added a comment - Github user okram commented on the pull request: https://github.com/apache/incubator-tinkerpop/pull/230#issuecomment-186256034 So weird. Again, overnight. ``` [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Apache TinkerPop .................................. SUCCESS [5.222s] [INFO] Apache TinkerPop :: Gremlin Shaded ................ SUCCESS [2.530s] [INFO] Apache TinkerPop :: Gremlin Core .................. SUCCESS [38.401s] [INFO] Apache TinkerPop :: Gremlin Test .................. SUCCESS [11.325s] [INFO] Apache TinkerPop :: Gremlin Groovy ................ SUCCESS [32.358s] [INFO] Apache TinkerPop :: Gremlin Groovy Test ........... SUCCESS [6.013s] [INFO] Apache TinkerPop :: TinkerGraph Gremlin ........... SUCCESS [3:15.039s] [INFO] Apache TinkerPop :: Hadoop Gremlin ................ SUCCESS [5:03.753s] [INFO] Apache TinkerPop :: Spark Gremlin ................. SUCCESS [10:01.122s] [INFO] Apache TinkerPop :: Giraph Gremlin ................ SUCCESS [2:24:33.823s] [INFO] Apache TinkerPop :: Neo4j Gremlin ................. SUCCESS [17:40.001s] [INFO] Apache TinkerPop :: Gremlin Driver ................ SUCCESS [8.919s] [INFO] Apache TinkerPop :: Gremlin Server ................ SUCCESS [10:56.373s] [INFO] Apache TinkerPop :: Gremlin Console ............... SUCCESS [1:08.209s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 3:14:23.560s [INFO] Finished at: Thu Feb 18 21:43:01 MST 2016 [INFO] Final Memory: 94M/708M [INFO] ------------------------------------------------------------------------ ``` Here I will just comment out that test. If you guys can tell me what result of the query is in the console I can fix it later, but without it failing here, I have no idea.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user okram commented on the pull request:

        https://github.com/apache/incubator-tinkerpop/pull/230#issuecomment-186262915

        Here is what I get in the console for that query:

        ```
        gremlin> g = TinkerFactory.createModern().traversal().withComputer()
        ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], tinkergraphcomputer]
        gremlin> g.V().hasLabel("software").has("name", "ripple").pageRank(1.0).by(_.inE("created")).times(1).by("priors").in("created").union(_.both(), __.identity()).valueMap("name", "priors")
        ==>[name:[marko], priors:[0.0]]
        ==>[name:[lop], priors:[0.0]]
        ==>[name:[josh], priors:[1.0]]
        ==>[name:[ripple], priors:[0.0]]
        gremlin>
        ```

        If someone who has a fail can tell me what they get that would help. However, again, I just commented out the test for now so I don't waste a day sitting here on an inanity.

        Show
        githubbot ASF GitHub Bot added a comment - Github user okram commented on the pull request: https://github.com/apache/incubator-tinkerpop/pull/230#issuecomment-186262915 Here is what I get in the console for that query: ``` gremlin> g = TinkerFactory.createModern().traversal().withComputer() ==>graphtraversalsource[tinkergraph [vertices:6 edges:6] , tinkergraphcomputer] gremlin> g.V().hasLabel("software").has("name", "ripple").pageRank(1.0).by(_ .inE("created")).times(1).by("priors").in("created").union( _.both(), __.identity()).valueMap("name", "priors") ==>[name: [marko] , priors: [0.0] ] ==>[name: [lop] , priors: [0.0] ] ==>[name: [josh] , priors: [1.0] ] ==>[name: [ripple] , priors: [0.0] ] gremlin> ``` If someone who has a fail can tell me what they get that would help. However, again, I just commented out the test for now so I don't waste a day sitting here on an inanity.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user dkuppitz commented on the pull request:

        https://github.com/apache/incubator-tinkerpop/pull/230#issuecomment-186276300

        The error was only [reported by Travis](https://travis-ci.org/apache/incubator-tinkerpop/builds/110233907). I can't reproduce it on my local machine. I ran that particular query a few thousand times to see if this is going to be another random error, but the result seems to be consistent.

        • `mvn clean install`: passed
        • integration tests: passed

        I looked roughly over the code; looks good, but it's hard to grasp all the changes in 120 files, I could spend a whole day on it, hence ...

        VOTE: +0.5 (just to get at least one more reviewer for this PR)

        ```
        [INFO] Reactor Summary:
        [INFO]
        [INFO] Apache TinkerPop .................................. SUCCESS [10.549s]
        [INFO] Apache TinkerPop :: Gremlin Shaded ................ SUCCESS [5.610s]
        [INFO] Apache TinkerPop :: Gremlin Core .................. SUCCESS [1:05.941s]
        [INFO] Apache TinkerPop :: Gremlin Test .................. SUCCESS [18.247s]
        [INFO] Apache TinkerPop :: Gremlin Groovy ................ SUCCESS [59.668s]
        [INFO] Apache TinkerPop :: Gremlin Groovy Test ........... SUCCESS [14.663s]
        [INFO] Apache TinkerPop :: TinkerGraph Gremlin ........... SUCCESS [9:36.751s]
        [INFO] Apache TinkerPop :: Hadoop Gremlin ................ SUCCESS [12:24.137s]
        [INFO] Apache TinkerPop :: Spark Gremlin ................. SUCCESS [26:11.246s]
        [INFO] Apache TinkerPop :: Giraph Gremlin ................ SUCCESS [2:59:20.410s]
        [INFO] Apache TinkerPop :: Neo4j Gremlin ................. SUCCESS [11.686s]
        [INFO] Apache TinkerPop :: Gremlin Driver ................ SUCCESS [28.561s]
        [INFO] Apache TinkerPop :: Gremlin Server ................ SUCCESS [12:02.678s]
        [INFO] Apache TinkerPop :: Gremlin Console ............... SUCCESS [1:56.698s]
        [INFO] ------------------------------------------------------------------------
        [INFO] BUILD SUCCESS
        [INFO] ------------------------------------------------------------------------
        [INFO] Total time: 4:05:07.482s
        [INFO] Finished at: Fri Feb 19 16:56:42 CET 2016
        [INFO] Final Memory: 62M/688M
        [INFO] ------------------------------------------------------------------------
        ```

        ^^ That's for the previous version (before you excluded the test).

        Show
        githubbot ASF GitHub Bot added a comment - Github user dkuppitz commented on the pull request: https://github.com/apache/incubator-tinkerpop/pull/230#issuecomment-186276300 The error was only [reported by Travis] ( https://travis-ci.org/apache/incubator-tinkerpop/builds/110233907 ). I can't reproduce it on my local machine. I ran that particular query a few thousand times to see if this is going to be another random error, but the result seems to be consistent. `mvn clean install`: passed integration tests: passed I looked roughly over the code; looks good, but it's hard to grasp all the changes in 120 files, I could spend a whole day on it, hence ... VOTE: +0.5 (just to get at least one more reviewer for this PR) ``` [INFO] Reactor Summary: [INFO] [INFO] Apache TinkerPop .................................. SUCCESS [10.549s] [INFO] Apache TinkerPop :: Gremlin Shaded ................ SUCCESS [5.610s] [INFO] Apache TinkerPop :: Gremlin Core .................. SUCCESS [1:05.941s] [INFO] Apache TinkerPop :: Gremlin Test .................. SUCCESS [18.247s] [INFO] Apache TinkerPop :: Gremlin Groovy ................ SUCCESS [59.668s] [INFO] Apache TinkerPop :: Gremlin Groovy Test ........... SUCCESS [14.663s] [INFO] Apache TinkerPop :: TinkerGraph Gremlin ........... SUCCESS [9:36.751s] [INFO] Apache TinkerPop :: Hadoop Gremlin ................ SUCCESS [12:24.137s] [INFO] Apache TinkerPop :: Spark Gremlin ................. SUCCESS [26:11.246s] [INFO] Apache TinkerPop :: Giraph Gremlin ................ SUCCESS [2:59:20.410s] [INFO] Apache TinkerPop :: Neo4j Gremlin ................. SUCCESS [11.686s] [INFO] Apache TinkerPop :: Gremlin Driver ................ SUCCESS [28.561s] [INFO] Apache TinkerPop :: Gremlin Server ................ SUCCESS [12:02.678s] [INFO] Apache TinkerPop :: Gremlin Console ............... SUCCESS [1:56.698s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 4:05:07.482s [INFO] Finished at: Fri Feb 19 16:56:42 CET 2016 [INFO] Final Memory: 62M/688M [INFO] ------------------------------------------------------------------------ ``` ^^ That's for the previous version (before you excluded the test).
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user twilmes commented on the pull request:

        https://github.com/apache/incubator-tinkerpop/pull/230#issuecomment-186282882

        Reproduced the failure b/f your last push on my linux box and that it was fixed after a pull. Looks good.

        VOTE +1

        Show
        githubbot ASF GitHub Bot added a comment - Github user twilmes commented on the pull request: https://github.com/apache/incubator-tinkerpop/pull/230#issuecomment-186282882 Reproduced the failure b/f your last push on my linux box and that it was fixed after a pull. Looks good. VOTE +1
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user spmallette commented on the pull request:

        https://github.com/apache/incubator-tinkerpop/pull/230#issuecomment-186285249

        All good from me at this point - I'm good after the fix:

        VOTE +1

        Show
        githubbot ASF GitHub Bot added a comment - Github user spmallette commented on the pull request: https://github.com/apache/incubator-tinkerpop/pull/230#issuecomment-186285249 All good from me at this point - I'm good after the fix: VOTE +1
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user okram commented on the pull request:

        https://github.com/apache/incubator-tinkerpop/pull/230#issuecomment-186285476

        Dope. I guess code reviews are important..........NOT.

        Show
        githubbot ASF GitHub Bot added a comment - Github user okram commented on the pull request: https://github.com/apache/incubator-tinkerpop/pull/230#issuecomment-186285476 Dope. I guess code reviews are important..........NOT.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user asfgit closed the pull request at:

        https://github.com/apache/incubator-tinkerpop/pull/230

        Show
        githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/incubator-tinkerpop/pull/230

          People

          • Assignee:
            okram Marko A. Rodriguez
            Reporter:
            okram Marko A. Rodriguez
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development