Solr
  1. Solr
  2. SOLR-594

StopFilterFactory, SynonymFilterFactory and EnglishProterFilterFactory not backwards compatible because of inform

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Won't Fix
    • Affects Version/s: 1.3
    • Fix Version/s: 1.3
    • Component/s: None
    • Labels:
      None

      Description

      Direct construction of StopWordFilter is not backwards compatible between 1.2 and 1.3. Here is some test code that throws a null pointer exception in 1.3 but that functions correctly in 1.2.

      TokenizerFactory tokenizer = new WhitespaceTokenizerFactory();

      Map<String, String> args = new HashMap<String, String>();
      args.put("ignoreCase", "true");
      args.put("words", "stopwords.txt");
      StopFilterFactory stopFilter = new StopFilterFactory();
      stopFilter.init(args);

      args = new HashMap<String, String>();
      args.put("generateWordParts", "1");
      args.put("generateNumberParts", "1");
      args.put("catenateWords", "0");
      args.put("catenateNumbers", "0");
      args.put("catenateAll", "0");
      WordDelimiterFilterFactory wordFilter = new WordDelimiterFilterFactory();
      wordFilter.init(args);

      TokenFilterFactory[] filters = new TokenFilterFactory[]

      {stopFilter, wordFilter }

      ;
      TokenizerChain pipeline =TokenizerChain(tokenizer, filters);

      /*** throws a null pointer exception in 1.3: ***/
      boolean onlyStopWords = pipeline.tokenStream(null, new StringReader(query)).next() == null;

      Hoss commented thusly in the solr forums (including a workaround):

      The short answer is: right after you call "stopFilter.init(args)" call
      "stopFilter.inform(solrCore.getSolrConfig().getResourceLoader());"

      This is an interesting use case that wasn't really considered when we
      switched away from using hte SolrCore singlton and the the
      ResourceLoaderAware interface was added. we made sure things would still
      work for people who had their own custom Analysis Factories, but some of
      the functionality in existing Factories was moved from the init() method
      to inform() ... which means the classes aren't technically backwards
      compatibly for people doing what you're doing: constructing them directly.

      When I have some more time, i'll spin up a thread on solr-dev to discuss
      what we should do about this – n the mean time feel free to file a bug
      that StopFilter isn't backwards compatible.

        Issue Links

          Activity

          Hide
          Hoss Man added a comment -

          (Updating title to reflect full scope of issue)

          Original solr-user thread pointing out problem...
          http://www.nabble.com/NullPointerException-at-lucene.analysis.StopFilter-with-1.3-to17564627.html#a17564627

          Discussion on solr-dev about how to deal with this...
          http://www.nabble.com/3-TokenFilter-factories-not-compatible-with-1.2-to17658628.html#a17658628

          ...current consensus seems to be that the best approach is...

          > 3) Documentation and Education
          > Since this wasn't exactly a use case we ever advertised, we could punt on
          > the problem by putting a disclaimer in the CAHNGES.txt that ayone directly
          > constructing those 3 classes should explicitly call inform() on the
          > instances after calling init.
          >
          >
          > #3 is obviously the simplest approach as developers, and to be quite honest:
          > probably impacts the fewest total number of people (since there are probably
          > very few people constructing Factory instances themselves)

          but first we're going to try and get some feedback from solr-user to verify that this really will only impact a small population of users.

          Show
          Hoss Man added a comment - (Updating title to reflect full scope of issue) Original solr-user thread pointing out problem... http://www.nabble.com/NullPointerException-at-lucene.analysis.StopFilter-with-1.3-to17564627.html#a17564627 Discussion on solr-dev about how to deal with this... http://www.nabble.com/3-TokenFilter-factories-not-compatible-with-1.2-to17658628.html#a17658628 ...current consensus seems to be that the best approach is... > 3) Documentation and Education > Since this wasn't exactly a use case we ever advertised, we could punt on > the problem by putting a disclaimer in the CAHNGES.txt that ayone directly > constructing those 3 classes should explicitly call inform() on the > instances after calling init. > > > #3 is obviously the simplest approach as developers, and to be quite honest: > probably impacts the fewest total number of people (since there are probably > very few people constructing Factory instances themselves) but first we're going to try and get some feedback from solr-user to verify that this really will only impact a small population of users.
          Hide
          Hoss Man added a comment -

          No negative response from community regarding leaving this incompatibility in 1.3 and documenting how to fix it for the few people it might affect. added note regarding this to CHANGES.txt

          Committed revision 677034.

          Show
          Hoss Man added a comment - No negative response from community regarding leaving this incompatibility in 1.3 and documenting how to fix it for the few people it might affect. added note regarding this to CHANGES.txt Committed revision 677034.

            People

            • Assignee:
              Hoss Man
              Reporter:
              Ronald Braun
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development