Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-8436

Realtime-get should support filters

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 5.4
    • Fix Version/s: 6.0
    • Component/s: None
    • Labels:
      None

      Description

      RTG currently ignores filters. There are probably other use-cases for RTG and filters, but one that comes to mind is security filters.

      1. SOLR-8436.patch
        13 kB
        Yonik Seeley
      2. SOLR-8436.patch
        11 kB
        Yonik Seeley
      3. SOLR-8436.patch
        8 kB
        Yonik Seeley
      4. SOLR-8436.patch
        7 kB
        Yonik Seeley

        Activity

        Hide
        yseeley@gmail.com Yonik Seeley added a comment -

        I can take a crack at adding this functionality...

        Show
        yseeley@gmail.com Yonik Seeley added a comment - I can take a crack at adding this functionality...
        Hide
        yseeley@gmail.com Yonik Seeley added a comment -

        Here's a draft patch that shows the approach of making sure a new realtime reader will be used if an add is found in the update log. Still need to apply the filters now.

        Show
        yseeley@gmail.com Yonik Seeley added a comment - Here's a draft patch that shows the approach of making sure a new realtime reader will be used if an add is found in the update log. Still need to apply the filters now.
        Hide
        yseeley@gmail.com Yonik Seeley added a comment -

        Here's a new patch with an efficient filtering implementation... it goes straight to the segment where the ID was found and then tries to advance the filter to that ID only.

        Show
        yseeley@gmail.com Yonik Seeley added a comment - Here's a new patch with an efficient filtering implementation... it goes straight to the segment where the ID was found and then tries to advance the filter to that ID only.
        Hide
        yseeley@gmail.com Yonik Seeley added a comment -

        Here's a new patch with working tests. All current tests pass as well.

        Show
        yseeley@gmail.com Yonik Seeley added a comment - Here's a new patch with working tests. All current tests pass as well.
        Hide
        yseeley@gmail.com Yonik Seeley added a comment -

        Updated patch that improves the tests and adds filtering into the RTG stress test. Now to loop the test for a while and make sure it's all air-tight...

        Show
        yseeley@gmail.com Yonik Seeley added a comment - Updated patch that improves the tests and adds filtering into the RTG stress test. Now to loop the test for a while and make sure it's all air-tight...
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1721585 from Yonik Seeley in branch 'dev/trunk'
        [ https://svn.apache.org/r1721585 ]

        SOLR-8436: filters for realtime-get

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1721585 from Yonik Seeley in branch 'dev/trunk' [ https://svn.apache.org/r1721585 ] SOLR-8436 : filters for realtime-get
        Hide
        dsmiley David Smiley added a comment -

        I was looking at the loop you have on the filters and advancing each scorer one by one... Couldn't you fill a BooleanQuery of FILTER clauses and more simply do this on one Query and thus also leverage the logic BooleanQuery has on costs & two-phase iterators?

        Show
        dsmiley David Smiley added a comment - I was looking at the loop you have on the filters and advancing each scorer one by one... Couldn't you fill a BooleanQuery of FILTER clauses and more simply do this on one Query and thus also leverage the logic BooleanQuery has on costs & two-phase iterators?
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1721593 from Yonik Seeley in branch 'dev/trunk'
        [ https://svn.apache.org/r1721593 ]

        SOLR-8436: remove nocommit

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1721593 from Yonik Seeley in branch 'dev/trunk' [ https://svn.apache.org/r1721593 ] SOLR-8436 : remove nocommit
        Hide
        yseeley@gmail.com Yonik Seeley added a comment -

        Couldn't you fill a BooleanQuery of FILTER clauses

        Very common case will be just a single filter, plus boolean query has more overhead these days - not sure if it would matter or be worth it to try and match a single doc. I'm not against it either, so feel free to take it if you want to make that change.

        Show
        yseeley@gmail.com Yonik Seeley added a comment - Couldn't you fill a BooleanQuery of FILTER clauses Very common case will be just a single filter, plus boolean query has more overhead these days - not sure if it would matter or be worth it to try and match a single doc. I'm not against it either, so feel free to take it if you want to make that change.

          People

          • Assignee:
            yseeley@gmail.com Yonik Seeley
            Reporter:
            yseeley@gmail.com Yonik Seeley
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development