Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-7377

SOLR Streaming Expressions

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • None
    • 5.2, 6.0
    • clients - java
    • None

    Description

      It would be beneficial to add an expression-based interface to Streaming API described in SOLR-7082. Right now that API requires streaming requests to come in from clients as serialized bytecode of the streaming classes. The suggestion here is to support string expressions which describe the streaming operations the client wishes to perform.

      search(collection1, q=*:*, fl="id,fieldA,fieldB", sort="fieldA asc")
      

      With this syntax in mind, one can now express arbitrarily complex stream queries with a single string.

      // merge two distinct searches together on common fields
      merge(
        search(collection1, q="id:(0 3 4)", fl="id,a_s,a_i,a_f", sort="a_f asc, a_s asc"),
        search(collection2, q="id:(1 2)", fl="id,a_s,a_i,a_f", sort="a_f asc, a_s asc"),
        on="a_f asc, a_s asc")
      
      // find top 20 unique records of a search
      top(
        n=20,
        unique(
          search(collection1, q=*:*, fl="id,a_s,a_i,a_f", sort="a_f desc"),
          over="a_f desc"),
        sort="a_f desc")
      

      The syntax would support
      1. Configurable expression names (eg. via solrconfig.xml one can map "unique" to a class implementing a Unique stream class) This allows users to build their own streams and use as they wish.
      2. Named parameters (of both simple and expression types)
      3. Unnamed, type-matched parameters (to support requiring N streams as arguments to another stream)
      4. Positional parameters

      The main goal here is to make streaming as accessible as possible and define a syntax for running complex queries across large distributed systems.

      Attachments

        1. SOLR-7377.patch
          297 kB
          Joel Bernstein
        2. SOLR-7377.patch
          286 kB
          Joel Bernstein
        3. SOLR-7377.patch
          287 kB
          Joel Bernstein
        4. SOLR-7377.patch
          286 kB
          Joel Bernstein
        5. SOLR-7377.patch
          290 kB
          Joel Bernstein
        6. SOLR-7377.patch
          286 kB
          Joel Bernstein
        7. SOLR-7377.patch
          275 kB
          Joel Bernstein
        8. SOLR-7377.patch
          274 kB
          Dennis Gove
        9. SOLR-7377.patch
          274 kB
          Dennis Gove
        10. SOLR-7377.patch
          263 kB
          Dennis Gove

        Issue Links

          Activity

            People

              Unassigned Unassigned
              dpgove Dennis Gove
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: