Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-7513

Add Equalitors to Streaming Expressions

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 6.0
    • Fix Version/s: None
    • Component/s: clients - java
    • Labels:
      None

      Description

      Right now all streams use the Comparator<Tuple> interface to compare tuples. The Comparator interface will tell you if tupleA is before, after, or equal to tupleB. This is great for most streams as they use this logic when combining multiple streams together. However, some streams only care about the equality of two tuples and the less/greater than logic is unnecessary.

      This depends on SOLR-7377.

      This patch is to introduce a new interface into streaming expressions called Equalitor<Tuple> which will return if two tuples are equal. The benefit here is that the expressions for streams using Equalitor instead of Comparator can omit the ordering part.

      unique(somestream, over="fieldA asc, fieldB desc")
      

      can become

      unique(somestream, over="fieldA,fieldB")
      

      The added benefit is that this will set us up with simplier expressions for joins (hash, merge, inner, outer, etc...) as those only care about equality.

      By adding this as an interface we make no assumptions about what it means to be equal, just that some implementation needs to exist adhering to the Equalitor<Tuple> interface which will determine if two tuples are logically equal.

      We do define at least one concrete class which checks for equality but that does not preclude others from adding additional concrete classes with their own logic in place.

      1. SOLR-7513.patch
        73 kB
        Joel Bernstein
      2. SOLR-7513.patch
        70 kB
        Joel Bernstein
      3. SOLR-7513.patch
        19 kB
        Dennis Gove
      4. SOLR-7513.patch
        19 kB
        Dennis Gove

        Activity

        Hide
        dpgove Dennis Gove added a comment -

        I'm pretty sure I want to change this to instead use Java's BiPredicate interface

        https://docs.oracle.com/javase/8/docs/api/java/util/function/BiPredicate.html

        Show
        dpgove Dennis Gove added a comment - I'm pretty sure I want to change this to instead use Java's BiPredicate interface https://docs.oracle.com/javase/8/docs/api/java/util/function/BiPredicate.html
        Hide
        andyetitmoves Ramkumar Aiyengar added a comment -

        Keep in mind that branch_5x still has only Java 7 as a requirement..

        Show
        andyetitmoves Ramkumar Aiyengar added a comment - Keep in mind that branch_5x still has only Java 7 as a requirement..
        Hide
        dpgove Dennis Gove added a comment -

        Modified Equalitor interface to more closely mirror Java 8's BiPredicate. I'm not using BiPredicate because this should be back-ported into 5.2 and as such needs to be Java 7 compatible.

        Depends on SOLR-7377, SOLR-7524, and SOLR-7528.

        Show
        dpgove Dennis Gove added a comment - Modified Equalitor interface to more closely mirror Java 8's BiPredicate. I'm not using BiPredicate because this should be back-ported into 5.2 and as such needs to be Java 7 compatible. Depends on SOLR-7377 , SOLR-7524 , and SOLR-7528 .
        Hide
        joel.bernstein Joel Bernstein added a comment -

        New patch created with svn diff against current trunk. StreamingTests are passing.

        Show
        joel.bernstein Joel Bernstein added a comment - New patch created with svn diff against current trunk. StreamingTests are passing.
        Hide
        joel.bernstein Joel Bernstein added a comment -

        The latest patch also includes the work from SOLR-7528.

        Show
        joel.bernstein Joel Bernstein added a comment - The latest patch also includes the work from SOLR-7528 .
        Hide
        joel.bernstein Joel Bernstein added a comment - - edited

        First impression is that it all looks good. I like the StreamComparator introduced in SOLR-7528 and the Equalitors are a nice design. I'll review the changes and run all the tests.

        Show
        joel.bernstein Joel Bernstein added a comment - - edited First impression is that it all looks good. I like the StreamComparator introduced in SOLR-7528 and the Equalitors are a nice design. I'll review the changes and run all the tests.
        Hide
        joel.bernstein Joel Bernstein added a comment -

        New patch with all tests passing and precommit. Just needed to add the package-info.java file for the new eq package.

        Show
        joel.bernstein Joel Bernstein added a comment - New patch with all tests passing and precommit. Just needed to add the package-info.java file for the new eq package.
        Hide
        joel.bernstein Joel Bernstein added a comment -

        This patch looks good to me and we have some pretty good tests behind it. the Streaming API tests, the Streaming Expressions tests and the Parallel SQL tests all exercise this code base. I plan to commit this to trunk shortly.

        Show
        joel.bernstein Joel Bernstein added a comment - This patch looks good to me and we have some pretty good tests behind it. the Streaming API tests, the Streaming Expressions tests and the Parallel SQL tests all exercise this code base. I plan to commit this to trunk shortly.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1686481 from Joel Bernstein in branch 'dev/trunk'
        [ https://svn.apache.org/r1686481 ]

        SOLR-7513, SOLR-7528: Add Equalitors to Streaming Expressions

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1686481 from Joel Bernstein in branch 'dev/trunk' [ https://svn.apache.org/r1686481 ] SOLR-7513 , SOLR-7528 : Add Equalitors to Streaming Expressions
        Hide
        dpgove Dennis Gove added a comment -

        I appreciate the help with this, Joel. Thanks!

        Show
        dpgove Dennis Gove added a comment - I appreciate the help with this, Joel. Thanks!
        Hide
        joel.bernstein Joel Bernstein added a comment - - edited

        Dennis, Thanks for the great work!

        Show
        joel.bernstein Joel Bernstein added a comment - - edited Dennis, Thanks for the great work!
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1692325 from Joel Bernstein in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1692325 ]

        SOLR-7513, SOLR-7528: Add Equalitors to Streaming Expressions

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1692325 from Joel Bernstein in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1692325 ] SOLR-7513 , SOLR-7528 : Add Equalitors to Streaming Expressions
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1692330 from Joel Bernstein in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1692330 ]

        SOLR-7513, SOLR-7528: Revert 5x commit

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1692330 from Joel Bernstein in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1692330 ] SOLR-7513 , SOLR-7528 : Revert 5x commit
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1694906 from Joel Bernstein in branch 'dev/trunk'
        [ https://svn.apache.org/r1694906 ]

        SOLR-7513: Updated CHANGES.txt

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1694906 from Joel Bernstein in branch 'dev/trunk' [ https://svn.apache.org/r1694906 ] SOLR-7513 : Updated CHANGES.txt
        Hide
        joel.bernstein Joel Bernstein added a comment -

        Releasing in Solr 6

        Show
        joel.bernstein Joel Bernstein added a comment - Releasing in Solr 6

          People

          • Assignee:
            joel.bernstein Joel Bernstein
            Reporter:
            dpgove Dennis Gove
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development