Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-8125

Umbrella ticket for Streaming and SQL issues

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: SolrJ
    • Labels:
      None

      Description

      This is an umbrella ticket for tracking issues around the Streaming API, Streaming Expressions and Parallel SQL.

      Issues can be linked to this ticket and discussions about the road map can also happen on this ticket.

        Issue Links

          Activity

          Hide
          risdenk Kevin Risden added a comment -

          We should really close out this ticket and move the few remaining items to a new one. Its hard to tell whats done and not here.

          Show
          risdenk Kevin Risden added a comment - We should really close out this ticket and move the few remaining items to a new one. Its hard to tell whats done and not here.
          Hide
          joel.bernstein Joel Bernstein added a comment -

          Kevin Risden, nice improvements!

          Show
          joel.bernstein Joel Bernstein added a comment - Kevin Risden , nice improvements!
          Hide
          gerlowskija Jason Gerlowski added a comment -

          I'd also add the (maybe less important) SOLR-8184.

          (Mentioning this here as much for my own tracking as for anything else.)

          Show
          gerlowskija Jason Gerlowski added a comment - I'd also add the (maybe less important) SOLR-8184 . (Mentioning this here as much for my own tracking as for anything else.)
          Hide
          risdenk Kevin Risden added a comment -

          Joel Bernstein - The following tickets should be ready for review:

          Show
          risdenk Kevin Risden added a comment - Joel Bernstein - The following tickets should be ready for review: SOLR-8190 SOLR-8527 SOLR-8529
          Hide
          joel.bernstein Joel Bernstein added a comment -

          SOLR-7535 is definitely an important one. SOLR-7525 is also important and just needs a few more tests, including parallel tests.

          Show
          joel.bernstein Joel Bernstein added a comment - SOLR-7535 is definitely an important one. SOLR-7525 is also important and just needs a few more tests, including parallel tests.
          Hide
          gerlowskija Jason Gerlowski added a comment -

          Looking to help out with some of the streaming/sql work linked to on this JIRA. Trying to get more familiar with this part of the code. Is there anything that stands out as what-should-be-worked-on-next? Not sure if there's any sort of priority attached to the subtasks for this umbrella issue.

          If not, I plan on taking a stab at SOLR-7535 (Add UpdateStream API), as it seems like a good way to dive in. Happy to take suggestions if anyone thinks that it'd be better to work on something else first though.

          Show
          gerlowskija Jason Gerlowski added a comment - Looking to help out with some of the streaming/sql work linked to on this JIRA. Trying to get more familiar with this part of the code. Is there anything that stands out as what-should-be-worked-on-next? Not sure if there's any sort of priority attached to the subtasks for this umbrella issue. If not, I plan on taking a stab at SOLR-7535 (Add UpdateStream API), as it seems like a good way to dive in. Happy to take suggestions if anyone thinks that it'd be better to work on something else first though.
          Hide
          joel.bernstein Joel Bernstein added a comment -

          Sounds good. We've got a couple months for bug fixing and testing.

          I'll probably be focused on testing and documentation mainly after wrapping up the last few issues I mentioned.

          Show
          joel.bernstein Joel Bernstein added a comment - Sounds good. We've got a couple months for bug fixing and testing. I'll probably be focused on testing and documentation mainly after wrapping up the last few issues I mentioned.
          Hide
          dpgove Dennis Gove added a comment -

          SOLR-8409 is a bug I'd like to get into Solr 6. I'd hate to see this go out in a major.

          Show
          dpgove Dennis Gove added a comment - SOLR-8409 is a bug I'd like to get into Solr 6. I'd hate to see this go out in a major.
          Hide
          joel.bernstein Joel Bernstein added a comment -

          SOLR-7525 might also be nice to have.

          Show
          joel.bernstein Joel Bernstein added a comment - SOLR-7525 might also be nice to have.
          Hide
          joel.bernstein Joel Bernstein added a comment -

          Awesome. I just committed SOLR-8337.

          Show
          joel.bernstein Joel Bernstein added a comment - Awesome. I just committed SOLR-8337 .
          Hide
          dpgove Dennis Gove added a comment -

          I'm working on SOLR-7904 and should have a patch by tomorrow. I'd also like to get SOLR-8185 into Solr 6 if I can get it done. Will spend some time on it this weekend.

          Show
          dpgove Dennis Gove added a comment - I'm working on SOLR-7904 and should have a patch by tomorrow. I'd also like to get SOLR-8185 into Solr 6 if I can get it done. Will spend some time on it this weekend.
          Hide
          joel.bernstein Joel Bernstein added a comment -

          Looking at the tickets listed above, I'd like to get the following three done before Solr 6:

          SOLR-8285
          SOLR-8337
          SOLR-7904

          Show
          joel.bernstein Joel Bernstein added a comment - Looking at the tickets listed above, I'd like to get the following three done before Solr 6: SOLR-8285 SOLR-8337 SOLR-7904
          Hide
          erickerickson Erick Erickson added a comment -

          Kevin:

          Yep, exactly. It would also be good to look through the issues already linked to this one to see if there are any dups in your list, and for any new JIRAs you create link them to this ticket.

          Show
          erickerickson Erick Erickson added a comment - Kevin: Yep, exactly. It would also be good to look through the issues already linked to this one to see if there are any dups in your list, and for any new JIRAs you create link them to this ticket.
          Hide
          risdenk Kevin Risden added a comment - - edited

          Joel Bernstein I've had a little time to look through the Streaming and SQL items as well as prototype some stuff out. Here is a list of items that I think need addressing:

          • SQL Tests
            • add error condition tests for JDBC driver (SOLR-8179 and SOLR-8184)
            • add where clause tests
              • currently where clauses only support equality and field names on the left
            • improve tests to use try-with-resources around Connection, Statement, ResultSets
            • use assertEquals/assertTrue/etc instead of plain assert statements
          • Streaming Tests
            • implement closeable support on TupleStream (SOLR-8190)
              • currently in tests if there is a failure resources aren't closed properly
              • closeable enables try-with-resources instead of wrapping tests with try/finally blocks.
            • use assertEquals/assertTrue/etc instead of plain assert statements
          • Functionality
            • expand JDBC implementation to support DBVisualizer/SquirrelSQL (SOLR-8502)
              • needs to be broken down into multiple JIRAs most likely
              • requires populating more metadata in JDBC driver
              • requires passing schema/field names back with result set
              • prototype is here currently just seeing what is possible: https://github.com/risdenk/lucene-solr/tree/expand-jdbc
            • JDBC Support for Authentication (SOLR-8213)
          • Minor
            • cleanup Streaming/SQL classes to use explicit types instead of plain maps/lists.

          What is the best way to go about these? Create JIRAs and submit patches?

          Show
          risdenk Kevin Risden added a comment - - edited Joel Bernstein I've had a little time to look through the Streaming and SQL items as well as prototype some stuff out. Here is a list of items that I think need addressing: SQL Tests add error condition tests for JDBC driver ( SOLR-8179 and SOLR-8184 ) add where clause tests currently where clauses only support equality and field names on the left improve tests to use try-with-resources around Connection, Statement, ResultSets use assertEquals/assertTrue/etc instead of plain assert statements Streaming Tests implement closeable support on TupleStream ( SOLR-8190 ) currently in tests if there is a failure resources aren't closed properly closeable enables try-with-resources instead of wrapping tests with try/finally blocks. use assertEquals/assertTrue/etc instead of plain assert statements Functionality expand JDBC implementation to support DBVisualizer/SquirrelSQL ( SOLR-8502 ) needs to be broken down into multiple JIRAs most likely requires populating more metadata in JDBC driver requires passing schema/field names back with result set prototype is here currently just seeing what is possible: https://github.com/risdenk/lucene-solr/tree/expand-jdbc JDBC Support for Authentication ( SOLR-8213 ) Minor cleanup Streaming/SQL classes to use explicit types instead of plain maps/lists. What is the best way to go about these? Create JIRAs and submit patches?

            People

            • Assignee:
              Unassigned
              Reporter:
              joel.bernstein Joel Bernstein
            • Votes:
              1 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:

                Development