Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-10404

The "fetch" streaming expression should escape join values in generated query

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 6.5
    • Fix Version/s: 6.5.1
    • Component/s: None
    • Security Level: Public (Default Security Level. Issues are Public)
    • Labels:
      None

      Description

      The "fetch" streaming expression joins data from another collection. Example:
      expr=fetch(collection,search(...), on="fieldA=fieldB"
      Internally, it does this by building a Solr query that looks like fieldB:(value1 value2 value3). But those values are not escaped; they should be. See FetchStream.java line 233. The ramification is that, for example, if a value contains a colon, then this isn't going to work.

        Activity

        Hide
        dsmiley David Smiley added a comment -

        Here's a patch that fixes the issue. It also sizes some lists & buffers better, and it generates a query that won't thrash the queryCache. I also ensured even the first value has a space before it to avoid a sneaky bug/trick to avoid the equivalent of a Solr query injection in the value. I've used that to "hack" known-Solr installations before – exercise to the reader Granted in this case it would be exceptionally odd based on how I see the "fetch" feature being used.

        It would be nice to use the "terms" QParser but unfortunately there is no escaping mechanism for the delimiter

        Should the generated query include a collapse on the field? Without it if multiple records have the value then the user might get multiple records for some values and no records back for others because "rows" is set. If we don't want a collapse (usually needless overhead?) then we ought to throw an exception if the numFound exceeds the rows.

        IMO, StreamExpressionTest is too big – it should be split up. And add useful test utility methods to avoid mundane repetition... similar to Lucene's BaseTokenStreamTestCase utility which is conceptually similar.

        Show
        dsmiley David Smiley added a comment - Here's a patch that fixes the issue. It also sizes some lists & buffers better, and it generates a query that won't thrash the queryCache. I also ensured even the first value has a space before it to avoid a sneaky bug/trick to avoid the equivalent of a Solr query injection in the value. I've used that to "hack" known-Solr installations before – exercise to the reader Granted in this case it would be exceptionally odd based on how I see the "fetch" feature being used. It would be nice to use the "terms" QParser but unfortunately there is no escaping mechanism for the delimiter Should the generated query include a collapse on the field? Without it if multiple records have the value then the user might get multiple records for some values and no records back for others because "rows" is set. If we don't want a collapse (usually needless overhead?) then we ought to throw an exception if the numFound exceeds the rows. IMO, StreamExpressionTest is too big – it should be split up. And add useful test utility methods to avoid mundane repetition... similar to Lucene's BaseTokenStreamTestCase utility which is conceptually similar.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit cb9f151db4b5ad5c5f581b6b8cf2e5916ddb0f35 in lucene-solr's branch refs/heads/master from David Smiley
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=cb9f151 ]

        SOLR-10404: fetch() streaming expression: escape values in generated query.

        Show
        jira-bot ASF subversion and git services added a comment - Commit cb9f151db4b5ad5c5f581b6b8cf2e5916ddb0f35 in lucene-solr's branch refs/heads/master from David Smiley [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=cb9f151 ] SOLR-10404 : fetch() streaming expression: escape values in generated query.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 048705e0e47ac04b6914e73c7ec3cc8daa12f2e0 in lucene-solr's branch refs/heads/branch_6x from David Smiley
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=048705e ]

        SOLR-10404: fetch() streaming expression: escape values in generated query.

        (cherry picked from commit cb9f151)

        Show
        jira-bot ASF subversion and git services added a comment - Commit 048705e0e47ac04b6914e73c7ec3cc8daa12f2e0 in lucene-solr's branch refs/heads/branch_6x from David Smiley [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=048705e ] SOLR-10404 : fetch() streaming expression: escape values in generated query. (cherry picked from commit cb9f151)
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 55eb30cc9d77196227d8e9d86aae77a55545f869 in lucene-solr's branch refs/heads/branch_6_5 from David Smiley
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=55eb30c ]

        SOLR-10404: fetch() streaming expression: escape values in generated query.

        (cherry picked from commit 048705e)

        Show
        jira-bot ASF subversion and git services added a comment - Commit 55eb30cc9d77196227d8e9d86aae77a55545f869 in lucene-solr's branch refs/heads/branch_6_5 from David Smiley [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=55eb30c ] SOLR-10404 : fetch() streaming expression: escape values in generated query. (cherry picked from commit 048705e)

          People

          • Assignee:
            dsmiley David Smiley
            Reporter:
            dsmiley David Smiley
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development