Solr
  1. Solr
  2. SOLR-2352

TermVectorComponent fails with "Undefined Field" errors for score, *, or any Solr 4x psuedo-fields used in the "fl" param.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.1
    • Fix Version/s: 4.0-ALPHA
    • Labels:
      None
    • Environment:

      Ubuntu 10.04/Arch solr 3.x branch r1058326

      Description

      When searching using the term vector components and setting fl=*,score the result is a http 400 error 'undefined field: *'. If you disable the tvc the search works properly.

      Example bad request...

      http://localhost:8983/solr/select/?qt=tvrh&q=includes:[*+TO+*]&fl=*

      3.1 stack trace:

      SEVERE: org.apache.solr.common.SolrException: undefined field: *
         at org.apache.solr.handler.component.TermVectorComponent.process(TermVectorComponent.java:142)
      ...
      

      The work around is to explicitly use the tv.fl param when using psuedo-fields in the fl...

      http://localhost:8983/solr/select/?qt=tvrh&q=includes:[*+TO+*]&fl=*&tv.fl=includes

        Issue Links

          Activity

          Hide
          Robert Muir added a comment -

          Moving out non-bug, non-test, unassigned issues from 3.1

          see http://www.lucidimagination.com/search/document/e5618bfdbfa8dcf6

          Show
          Robert Muir added a comment - Moving out non-bug, non-test, unassigned issues from 3.1 see http://www.lucidimagination.com/search/document/e5618bfdbfa8dcf6
          Hide
          Jan Rasehorn added a comment - - edited

          This issue breaks SOLR-2444, since solr returns the same error for transformers and functions used in the fl-parameter. I solved the problem by disabling the term vector component in solrconfig.xml.

          I am using Solr 4 Trunk nightly build as of 16th Dec 2011.

          Show
          Jan Rasehorn added a comment - - edited This issue breaks SOLR-2444 , since solr returns the same error for transformers and functions used in the fl-parameter. I solved the problem by disabling the term vector component in solrconfig.xml. I am using Solr 4 Trunk nightly build as of 16th Dec 2011.
          Hide
          Hoss Man added a comment -

          revised description & summary to clarify full scope of problem and mention work around

          Show
          Hoss Man added a comment - revised description & summary to clarify full scope of problem and mention work around
          Hide
          Hoss Man added a comment -

          patch with tests that fixes the core bug ... ReturnFields is used by TVC to parse the "fl" and only real fields found in it are used ... which fixes the "* is not a field" bug and brings the functionality back on par with past releases if any globs, functions or transformers are used in fl.

          But looking at the code i think there's a lot of room for improvement:

          • no partial globs are supported in fl or tv.fl (ie: "fl=*_t"
          • per field override options are ignored when all TV are returned for all fields (either because of "fl=*" or no fl/tv.fl
          • code structure is wonky

          ...the last item seemingly a relic from when the code use to use the "TermVectorMapper" interface to walk the vectors the various fields, and used diff code paths depending on wether "all" fields were requested, or just specific ones.

          I'm going to try and take a stab at refactoring this to only ever use "reader.getTermVectors(docId)" (combined with ReturnFields.wantField) .. but if i don't get to it, this patch should still be good to go for 4.0

          Show
          Hoss Man added a comment - patch with tests that fixes the core bug ... ReturnFields is used by TVC to parse the "fl" and only real fields found in it are used ... which fixes the "* is not a field" bug and brings the functionality back on par with past releases if any globs, functions or transformers are used in fl. But looking at the code i think there's a lot of room for improvement: no partial globs are supported in fl or tv.fl (ie: "fl=*_t" per field override options are ignored when all TV are returned for all fields (either because of "fl=*" or no fl/tv.fl code structure is wonky ...the last item seemingly a relic from when the code use to use the "TermVectorMapper" interface to walk the vectors the various fields, and used diff code paths depending on wether "all" fields were requested, or just specific ones. I'm going to try and take a stab at refactoring this to only ever use "reader.getTermVectors(docId)" (combined with ReturnFields.wantField) .. but if i don't get to it, this patch should still be good to go for 4.0
          Hide
          Robert Muir added a comment -

          ...the last item seemingly a relic from when the code use to use the "TermVectorMapper" interface to walk the vectors the various fields, and used diff code paths depending on wether "all" fields were requested, or just specific ones.

          I didnt look at the patch, or the issue, but maybe in the case only specific fields are returned you could just wrap
          the Fields returned by getTermVectors with a "FilteredFields" so you only have one codepath:

          http://svn.apache.org/repos/asf/lucene/dev/trunk/lucene/test-framework/src/java/org/apache/lucene/index/FieldFilterAtomicReader.java

          Show
          Robert Muir added a comment - ...the last item seemingly a relic from when the code use to use the "TermVectorMapper" interface to walk the vectors the various fields, and used diff code paths depending on wether "all" fields were requested, or just specific ones. I didnt look at the patch, or the issue, but maybe in the case only specific fields are returned you could just wrap the Fields returned by getTermVectors with a "FilteredFields" so you only have one codepath: http://svn.apache.org/repos/asf/lucene/dev/trunk/lucene/test-framework/src/java/org/apache/lucene/index/FieldFilterAtomicReader.java
          Hide
          Hoss Man added a comment -

          Committed revision 1349012. - trunk
          Committed revision 1349013. - 4x

          Show
          Hoss Man added a comment - Committed revision 1349012. - trunk Committed revision 1349013. - 4x
          Hide
          Hoss Man added a comment -

          my weekend was busier then i expected, so i went ahead and committed the patch to fix the bug, and spun off SOLR-3537 for future improvements.

          Show
          Hoss Man added a comment - my weekend was busier then i expected, so i went ahead and committed the patch to fix the bug, and spun off SOLR-3537 for future improvements.

            People

            • Assignee:
              Hoss Man
              Reporter:
              Jed Glazner
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development