Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-8972

Add GraphHandler and GraphMLResponseWriter to support graph visualizations

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 6.1
    • Component/s: None
    • Labels:
      None

      Description

      SOLR-8925 is shaping up nicely. It would be great if Solr could support outputting graphs in GraphML. This will allow users to visualize their graphs in a number of graph visualization tools (NodeXL, Gephi, Tulip etc...). This ticket will create a new Graph handler which will take a Streaming Expression graph traversal and output GraphML. A new GraphMLResponseWriter will handle the GraphML formatting. In future releases we can consider supporting other graph formats.

      1. GraphHandler.java
        10 kB
        Joel Bernstein
      2. GraphMLResponseWriter.java
        4 kB
        Joel Bernstein
      3. SOLR-8972.patch
        27 kB
        Joel Bernstein
      4. SOLR-8972.patch
        26 kB
        Joel Bernstein
      5. SOLR-8972.patch
        22 kB
        Joel Bernstein

        Activity

        Hide
        joel.bernstein Joel Bernstein added a comment - - edited

        Initial implementations of the GraphHandler and GraphMLResponsWriter attached. The GraphHandler simply takes a Streaming Expression compiles it and initializes a Traversal object and adds it to the StreamContext. The TupleStream and Traversal objects are added to the request context. The GraphMLResponseWriter gets the TupleStream and Traversal from the request context, opens the stream and iterates the Tuples outputting the GraphML formatting for the nodes and edges. All the traversal logic is encapsulated inside of the TupleStream.

        Proper patches and test cases to follow.

        Show
        joel.bernstein Joel Bernstein added a comment - - edited Initial implementations of the GraphHandler and GraphMLResponsWriter attached. The GraphHandler simply takes a Streaming Expression compiles it and initializes a Traversal object and adds it to the StreamContext. The TupleStream and Traversal objects are added to the request context. The GraphMLResponseWriter gets the TupleStream and Traversal from the request context, opens the stream and iterates the Tuples outputting the GraphML formatting for the nodes and edges. All the traversal logic is encapsulated inside of the TupleStream. Proper patches and test cases to follow.
        Hide
        joel.bernstein Joel Bernstein added a comment -

        Sample syntax for calling the GraphHandler:

        http://localhost:8983/solr/collection/graph?expr=gatherNodes(...)

        The expression can be an arbitrarily complex nested gatherNodes (SOLR-8925) expression wrapped by other Streaming Expressions. The graph handler will be hardcoded to use the graphML response writer.

        Show
        joel.bernstein Joel Bernstein added a comment - Sample syntax for calling the GraphHandler: http://localhost:8983/solr/collection/graph?expr=gatherNodes( ...) The expression can be an arbitrarily complex nested gatherNodes ( SOLR-8925 ) expression wrapped by other Streaming Expressions. The graph handler will be hardcoded to use the graphML response writer.
        Hide
        joel.bernstein Joel Bernstein added a comment -

        Patch with functional GraphHandler and GraphMLResponseWriter. Patch includes test cases for the GraphMLResponseWriter. Still need to add tests for the GraphHandler.

        Show
        joel.bernstein Joel Bernstein added a comment - Patch with functional GraphHandler and GraphMLResponseWriter. Patch includes test cases for the GraphMLResponseWriter. Still need to add tests for the GraphHandler.
        Hide
        joel.bernstein Joel Bernstein added a comment -

        Added simple test for the GraphHandler

        Show
        joel.bernstein Joel Bernstein added a comment - Added simple test for the GraphHandler
        Hide
        joel.bernstein Joel Bernstein added a comment -

        Pre-commit is passing with the latest patch. I need to do some looking into the error handling with this and then I think it's getting pretty close to committable.

        Show
        joel.bernstein Joel Bernstein added a comment - Pre-commit is passing with the latest patch. I need to do some looking into the error handling with this and then I think it's getting pretty close to committable.
        Hide
        joel.bernstein Joel Bernstein added a comment - - edited

        Added some error handling to the GraphMLResponseWriter. Now it prints a stack trace to the client if it sees a error coming from the GraphHandler. The GraphHandler will log the error before passing the error over to the GraphResponseWriter.

        It's using a DummyErrorStream to wrap the error in a TupleStream. This is to keep things consistent with the StreamHandler and also in case future graph writers export in a json graph format and can use the DummyErrorStream to send a json error to the client.

        Show
        joel.bernstein Joel Bernstein added a comment - - edited Added some error handling to the GraphMLResponseWriter. Now it prints a stack trace to the client if it sees a error coming from the GraphHandler. The GraphHandler will log the error before passing the error over to the GraphResponseWriter. It's using a DummyErrorStream to wrap the error in a TupleStream. This is to keep things consistent with the StreamHandler and also in case future graph writers export in a json graph format and can use the DummyErrorStream to send a json error to the client.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit be1cb9a1cde4dd426305f22620734d018f21dd82 in lucene-solr's branch refs/heads/master from jbernste
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=be1cb9a ]

        SOLR-8972: Add GraphHandler and GraphMLResponseWriter to support graph visualizations

        Show
        jira-bot ASF subversion and git services added a comment - Commit be1cb9a1cde4dd426305f22620734d018f21dd82 in lucene-solr's branch refs/heads/master from jbernste [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=be1cb9a ] SOLR-8972 : Add GraphHandler and GraphMLResponseWriter to support graph visualizations
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 02eef8dffac0d9f3cda86bb3834a6cc769962ffb in lucene-solr's branch refs/heads/master from jbernste
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=02eef8d ]

        SOLR-8972: Update CHANGES.txt

        Show
        jira-bot ASF subversion and git services added a comment - Commit 02eef8dffac0d9f3cda86bb3834a6cc769962ffb in lucene-solr's branch refs/heads/master from jbernste [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=02eef8d ] SOLR-8972 : Update CHANGES.txt
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 928a3cf268b00e9589238adf08848d8eee7c83c0 in lucene-solr's branch refs/heads/branch_6x from jbernste
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=928a3cf ]

        SOLR-8972: Add GraphHandler and GraphMLResponseWriter to support graph visualizations

        Show
        jira-bot ASF subversion and git services added a comment - Commit 928a3cf268b00e9589238adf08848d8eee7c83c0 in lucene-solr's branch refs/heads/branch_6x from jbernste [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=928a3cf ] SOLR-8972 : Add GraphHandler and GraphMLResponseWriter to support graph visualizations
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 8b56f67adb4192793795b6351ef021cb5a4149ac in lucene-solr's branch refs/heads/branch_6x from jbernste
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=8b56f67 ]

        SOLR-8972: Update CHANGES.txt

        Show
        jira-bot ASF subversion and git services added a comment - Commit 8b56f67adb4192793795b6351ef021cb5a4149ac in lucene-solr's branch refs/heads/branch_6x from jbernste [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=8b56f67 ] SOLR-8972 : Update CHANGES.txt
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 66fc6aaf8d5241bc845c049742662dbf9a5b1840 in lucene-solr's branch refs/heads/master from jbernste
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=66fc6aa ]

        SOLR-8972: Fix MinimalSchemaTest and SolrCoreTest

        Show
        jira-bot ASF subversion and git services added a comment - Commit 66fc6aaf8d5241bc845c049742662dbf9a5b1840 in lucene-solr's branch refs/heads/master from jbernste [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=66fc6aa ] SOLR-8972 : Fix MinimalSchemaTest and SolrCoreTest
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit ed1db6d20a6dbd30ec0f2b3d2354eb4d61c630a9 in lucene-solr's branch refs/heads/branch_6x from jbernste
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ed1db6d ]

        SOLR-8972: Fix MinimalSchemaTest and SolrCoreTest

        Show
        jira-bot ASF subversion and git services added a comment - Commit ed1db6d20a6dbd30ec0f2b3d2354eb4d61c630a9 in lucene-solr's branch refs/heads/branch_6x from jbernste [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ed1db6d ] SOLR-8972 : Fix MinimalSchemaTest and SolrCoreTest
        Hide
        ctargett Cassandra Targett added a comment -

        Joel Bernstein: Is this done? If so, can we mark it as resolved?

        Show
        ctargett Cassandra Targett added a comment - Joel Bernstein : Is this done? If so, can we mark it as resolved?

          People

          • Assignee:
            joel.bernstein Joel Bernstein
            Reporter:
            joel.bernstein Joel Bernstein
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development