Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-7739

Lucene Classification Integration - UpdateRequestProcessor

    Details

    • Flags:
      Patch

      Description

      It would be nice to have an UpdateRequestProcessor to interact with the Lucene Classification Module and provide an easy way of auto classifying Solr Documents on indexing.

      Documentation will be provided with the patch
      A first design will be provided soon.

      1. SOLR-7739.1.patch
        1 kB
        Tommaso Teofili
      2. SOLR-7739.patch
        27 kB
        Alessandro Benedetti
      3. SOLR-7739.patch
        25 kB
        Alessandro Benedetti
      4. SOLR-7739.patch
        25 kB
        Alessandro Benedetti

        Issue Links

          Activity

          Hide
          alessandro.benedetti Alessandro Benedetti added a comment - - edited

          The Design of the Solution of this issue will require features from the linked issue.

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - - edited The Design of the Solution of this issue will require features from the linked issue.
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          Will follow documentation and the patch during the week end.

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - Will follow documentation and the patch during the week end.
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          First patch attached, it contains :

          • the implementation for the Update Request Processor, the Factory.
          • a Unit Test
          • Solr Test configuration for the Test
          • The Lucene Classification Dependency for the Solr build
          Show
          alessandro.benedetti Alessandro Benedetti added a comment - First patch attached, it contains : the implementation for the Update Request Processor, the Factory. a Unit Test Solr Test configuration for the Test The Lucene Classification Dependency for the Solr build
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          http://alexbenedetti.blogspot.co.uk/2015/07/solr-document-classification-part-1.html

          Documentation in the blog post, hopefully it is exhaustive.

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - http://alexbenedetti.blogspot.co.uk/2015/07/solr-document-classification-part-1.html Documentation in the blog post, hopefully it is exhaustive.
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          Java docs fixed

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - Java docs fixed
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          Any feedback guys ?

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - Any feedback guys ?
          Hide
          teofili Tommaso Teofili added a comment -

          Alessandro, thanks for your patch, I'll take a look and let you know shortly.

          Show
          teofili Tommaso Teofili added a comment - Alessandro, thanks for your patch, I'll take a look and let you know shortly.
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          Any news on this ?
          Cheers

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - Any news on this ? Cheers
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          After the update of the related patch , applying this patch is straightforward.
          Let me know for any feedback!

          Cheers

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - After the update of the related patch , applying this patch is straightforward. Let me know for any feedback! Cheers
          Hide
          daviddekleer David de Kleer added a comment -

          An easy way to classify text from within SOLR would be a very nice feature! Could you please take another look at this? I have tried to implement this patch but didn't succeed, because the patch underlying this patch (6631) got changed/updated in the meantime.

          Show
          daviddekleer David de Kleer added a comment - An easy way to classify text from within SOLR would be a very nice feature! Could you please take another look at this? I have tried to implement this patch but didn't succeed, because the patch underlying this patch (6631) got changed/updated in the meantime.
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          Hi David,
          tomorrow I will update this patch.
          And ping the committers for having a feedback !
          The lucene side has already been officially integrated !

          Cheers

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - Hi David, tomorrow I will update this patch. And ping the committers for having a feedback ! The lucene side has already been officially integrated ! Cheers
          Hide
          daviddekleer David de Kleer added a comment -

          Hi Alessandro,
          Thank you for your quick response, that's nice!
          With kind regards,
          David

          Show
          daviddekleer David de Kleer added a comment - Hi Alessandro, Thank you for your quick response, that's nice! With kind regards, David
          Hide
          daviddekleer David de Kleer added a comment - - edited

          By the way, wouldn't it be useful to provide confidence values/scores for classifications, so you're able to see how much confidence a classifier has in its predictions?

          David

          Show
          daviddekleer David de Kleer added a comment - - edited By the way, wouldn't it be useful to provide confidence values/scores for classifications, so you're able to see how much confidence a classifier has in its predictions? David
          Hide
          daviddekleer David de Kleer added a comment -

          Any updates on your update?

          David

          Show
          daviddekleer David de Kleer added a comment - Any updates on your update? David
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          Hi David,
          I quickly took a look and the problem was in the method signature not compliant.
          I will upload a quick fix soon.

          Related the confidence.
          The classifier already return a confidence score, this is something I plan to return with the Classification Request handler.
          The update processor is meant to work without human interaction, so it was not indexing any confidence score.
          Of course you are free to customise it!

          Cheers

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - Hi David, I quickly took a look and the problem was in the method signature not compliant. I will upload a quick fix soon. Related the confidence. The classifier already return a confidence score, this is something I plan to return with the Classification Request handler. The update processor is meant to work without human interaction, so it was not indexing any confidence score. Of course you are free to customise it! Cheers
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          Method signature updattest solrconfig updated

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - Method signature updattest solrconfig updated
          Hide
          daviddekleer David de Kleer added a comment -

          Awesome! I will test it soon! :-D

          David

          Show
          daviddekleer David de Kleer added a comment - Awesome! I will test it soon! :-D David
          Hide
          daviddekleer David de Kleer added a comment -

          Hi Alessandro,

          All right, I've managed to build SOLR from the Github mirror, with your patch applied. I thought you could use it like this?

          <!-- classification -->
          <updateRequestProcessorChain name="classification">
              <processor class="solr.ClassificationUpdateProcessorFactory">
                  <str name="inputFields">q_website</str>
                  <str name="classField">labels</str>
                  <str name="algorithm">bayes</str>
              </processor>
              <processor class="solr.LogUpdateProcessorFactory"/>
              <processor class="solr.RunUpdateProcessorFactory"/>
          </updateRequestProcessorChain>
          
          <requestHandler name="/update" class="solr.UpdateRequestHandler">
              <lst name="defaults">
                  <str name="update.chain">classification</str>
              </lst>
          </requestHandler>
          

          But how is it possible to train a model? I can't trace any activity of the classification updateRequestProcessorChain on an update (labels is a multivalued string btw, don't know if that is a problem). Or am I doing something wrong?

          With kind regards,

          David

          Show
          daviddekleer David de Kleer added a comment - Hi Alessandro, All right, I've managed to build SOLR from the Github mirror, with your patch applied. I thought you could use it like this? <!-- classification --> <updateRequestProcessorChain name= "classification" > <processor class= "solr.ClassificationUpdateProcessorFactory" > <str name= "inputFields" >q_website</str> <str name= "classField" >labels</str> <str name= "algorithm" >bayes</str> </processor> <processor class= "solr.LogUpdateProcessorFactory" /> <processor class= "solr.RunUpdateProcessorFactory" /> </updateRequestProcessorChain> <requestHandler name= "/update" class= "solr.UpdateRequestHandler" > <lst name= "defaults" > <str name= "update.chain" >classification</str> </lst> </requestHandler> But how is it possible to train a model? I can't trace any activity of the classification updateRequestProcessorChain on an update (labels is a multivalued string btw, don't know if that is a problem). Or am I doing something wrong? With kind regards, David
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          David,
          take a look to the documentation :

          http://alexbenedetti.blogspot.co.uk/2015/07/solr-document-classification-part-1.html

          You don't need to train any model.
          The index is your trained model.
          The multi valued thing should not be a problem.

          Cheers

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - David, take a look to the documentation : http://alexbenedetti.blogspot.co.uk/2015/07/solr-document-classification-part-1.html You don't need to train any model. The index is your trained model. The multi valued thing should not be a problem. Cheers
          Hide
          daviddekleer David de Kleer added a comment - - edited

          Thank you for your response! Your blogpost has already been my guidance for my adaptations of solrconfig.xml (and the reason I found out about this patch ) But it could be that I'm missing something.

          With kind regards,

          David

          Show
          daviddekleer David de Kleer added a comment - - edited Thank you for your response! Your blogpost has already been my guidance for my adaptations of solrconfig.xml (and the reason I found out about this patch ) But it could be that I'm missing something. With kind regards, David
          Hide
          daviddekleer David de Kleer added a comment - - edited

          Never mind, I added a custom field labels_auto to put the labels in and a print statement to be able to see the labels that were added. Now it's working! Or well, I can see that it works xD

          David

          Show
          daviddekleer David de Kleer added a comment - - edited Never mind, I added a custom field labels_auto to put the labels in and a print statement to be able to see the labels that were added. Now it's working! Or well, I can see that it works xD David
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          mmmm David, can you explain why it was not working and what was your difficulty ?
          The processor is going to assign the class to the document and set the field in the Solr document.
          The field set is the classField name configured in the solrconfig.
          It is a pretty simple configuration, could you explain what went wrong ?
          In this way we can improve the documentation.

          Cheers

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - mmmm David, can you explain why it was not working and what was your difficulty ? The processor is going to assign the class to the document and set the field in the Solr document. The field set is the classField name configured in the solrconfig. It is a pretty simple configuration, could you explain what went wrong ? In this way we can improve the documentation. Cheers
          Hide
          daviddekleer David de Kleer added a comment - - edited

          Hi Alessandro,

          It turned out there was no problem at all, it was more that I didn't see that labels were being added (or not) in SOLR's output/the web interface. That's why I added a print statement and a slight modification to the ClassificationUpdateProcessor, something like

          ...
          score = classificationResult.getScore();
          System.out.println("(ClassificationUpdateProcessor) Found class " + assignedClass + " with score " + score + " for this document.");
          doc.addField("label_" + assignedClass, score);
          ...
          

          for example. The print statement and the searchable score (below 1) gave me an indication that labels were really being added. So again, it turned out it wasn't really a problem. And thanks again for updating the patch!

          EDIT: In fact, all of this means that I just wanted to make the distinction between annotations and automatically added labels/categories a bit more clear.

          With kind regards,

          David

          Show
          daviddekleer David de Kleer added a comment - - edited Hi Alessandro, It turned out there was no problem at all, it was more that I didn't see that labels were being added (or not) in SOLR's output/the web interface. That's why I added a print statement and a slight modification to the ClassificationUpdateProcessor, something like ... score = classificationResult.getScore(); System .out.println( "(ClassificationUpdateProcessor) Found class " + assignedClass + " with score " + score + " for this document." ); doc.addField( "label_" + assignedClass, score); ... for example. The print statement and the searchable score (below 1) gave me an indication that labels were really being added. So again, it turned out it wasn't really a problem. And thanks again for updating the patch! EDIT: In fact, all of this means that I just wanted to make the distinction between annotations and automatically added labels/categories a bit more clear. With kind regards, David
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          Perfect ! Still I have no feedback from the community, it could be useful to get any feedback to improve the component.
          Or trying to understand why it is not good to have it included in Solr ...

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - Perfect ! Still I have no feedback from the community, it could be useful to get any feedback to improve the component. Or trying to understand why it is not good to have it included in Solr ...
          Hide
          joel.bernstein Joel Bernstein added a comment -

          I've been interested in this ticket, I just haven't had time to review it yet. I do think this is good feature for Solr and the company I work for, Alfresco, would also find this useful. I hope to carve out some time to review following the 6.0 release.

          Show
          joel.bernstein Joel Bernstein added a comment - I've been interested in this ticket, I just haven't had time to review it yet. I do think this is good feature for Solr and the company I work for, Alfresco, would also find this useful. I hope to carve out some time to review following the 6.0 release.
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          Joel, thank you very much.
          As you will see, it is a very simple contribution.
          But I am really curious to have some feedback to improve it and make it more suitable for real cases
          Let me know when you have time,

          Cheers

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - Joel, thank you very much. As you will see, it is a very simple contribution. But I am really curious to have some feedback to improve it and make it more suitable for real cases Let me know when you have time, Cheers
          Hide
          joel.bernstein Joel Bernstein added a comment -

          I'm also curious about whether LUCENE-6968 can be used with this ticket.

          Show
          joel.bernstein Joel Bernstein added a comment - I'm also curious about whether LUCENE-6968 can be used with this ticket.
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          Actually the algorithm is a parameter for the Solr Classification Update processor!
          We could design and develop a DocumentClassifier ( Lucene side) to use the LSH approach.
          Then simply switch to that algorithm in the solrconfig for the use cases we want!
          Potentially we could ask the author to contribute a DocumentClassifier based on that approach OR we could design and develop one.
          I agree it is interesting!

          Cao Manh Dat what do you think ?

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - Actually the algorithm is a parameter for the Solr Classification Update processor! We could design and develop a DocumentClassifier ( Lucene side) to use the LSH approach. Then simply switch to that algorithm in the solrconfig for the use cases we want! Potentially we could ask the author to contribute a DocumentClassifier based on that approach OR we could design and develop one. I agree it is interesting! Cao Manh Dat what do you think ?
          Hide
          teofili Tommaso Teofili added a comment -

          my bad not having had time to include Alessandro's patch so far.
          I plan to review it by the end of the week, if anyone has some spare cycle to overtake this of course feel free

          Show
          teofili Tommaso Teofili added a comment - my bad not having had time to include Alessandro's patch so far. I plan to review it by the end of the week, if anyone has some spare cycle to overtake this of course feel free
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          No worries Tommaso
          Take your time, any feedback is welcome to improve the patch !

          Cheers

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - No worries Tommaso Take your time, any feedback is welcome to improve the patch ! Cheers
          Hide
          teofili Tommaso Teofili added a comment - - edited

          I managed to have a brief look and I think this probably deserves either a separate module or anyway not to be in core as to me this sounds more like an extension that some Solr installations may or may not want to use e.g. like SolrCell, SolrUIMA, etc.
          Also the ant build fails as it seems the Lucene Classification module is not correctly imported il Solr Core module, I'll have a second look later today.

          Show
          teofili Tommaso Teofili added a comment - - edited I managed to have a brief look and I think this probably deserves either a separate module or anyway not to be in core as to me this sounds more like an extension that some Solr installations may or may not want to use e.g. like SolrCell, SolrUIMA, etc. Also the ant build fails as it seems the Lucene Classification module is not correctly imported il Solr Core module, I'll have a second look later today.
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          Hey Tommaso, thanks for the review !
          Really weird the build fails, pretty sure the only addition is the dependency to the lucene classification module in the commons-build.xml !
          I will check tomorrow why is not building anymore !

          Related the module location feel free to identify the proper location!
          Although I think this case is different from SolrUIMA as it is not an integration with an external framework, it is still only using an internal Lucene module !

          Cheers

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - Hey Tommaso, thanks for the review ! Really weird the build fails, pretty sure the only addition is the dependency to the lucene classification module in the commons-build.xml ! I will check tomorrow why is not building anymore ! Related the module location feel free to identify the proper location! Although I think this case is different from SolrUIMA as it is not an integration with an external framework, it is still only using an internal Lucene module ! Cheers
          Hide
          joel.bernstein Joel Bernstein added a comment -

          I haven't done a review yet, but it sounds like it would make sense in core. The patch seems fairly small and the functionality could be considered core to Solr's central use cases.

          Show
          joel.bernstein Joel Bernstein added a comment - I haven't done a review yet, but it sounds like it would make sense in core. The patch seems fairly small and the functionality could be considered core to Solr's central use cases.
          Hide
          joel.bernstein Joel Bernstein added a comment - - edited

          I mentioned the LSH work in another comment, but I'm also interested in Logistic Regression classification (SOLR-8492), for use cases like fraud detection. This also ties in with the TopicStream (SOLR-8588) work, where topics can be created based on document classifications.

          Show
          joel.bernstein Joel Bernstein added a comment - - edited I mentioned the LSH work in another comment, but I'm also interested in Logistic Regression classification ( SOLR-8492 ), for use cases like fraud detection. This also ties in with the TopicStream ( SOLR-8588 ) work, where topics can be created based on document classifications.
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          Joel, very interesting topics ! I will read more about them as soon as I have time, from the related patches.

          Related the Classification Processor, as you will notice is a very simple piece.
          I definitely would like to spend some time to improve it.

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - Joel, very interesting topics ! I will read more about them as soon as I have time, from the related patches. Related the Classification Processor, as you will notice is a very simple piece. I definitely would like to spend some time to improve it.
          Hide
          teofili Tommaso Teofili added a comment - - edited

          good points Joel Bernstein, I've attached Alessandro Benedetti's patch with a few changes in order to make the build (compile and test) succeed (keeping ClassificationUpdateProcessor in solr-core).
          ant documentation and ant documentation-lint also succeed.
          ant idea doesn't correctly set the dependencies for solr-core so that one has to manually add lucene-classification to the classpath and maybe it's the same for ant eclipse

          Show
          teofili Tommaso Teofili added a comment - - edited good points Joel Bernstein , I've attached Alessandro Benedetti 's patch with a few changes in order to make the build (compile and test) succeed (keeping ClassificationUpdateProcessor  in solr-core ). ant documentation and ant documentation-lint also succeed. ant idea doesn't correctly set the dependencies for solr-core so that one has to manually add lucene-classification to the classpath and maybe it's the same for ant eclipse
          Hide
          teofili Tommaso Teofili added a comment -

          After having had a closer look at the code I think it generally looks good.
          One thing I noticed is that the Query parameter to build Classifiers that use a subset of the index is missing, while it's ok for a first implementation I think that may be good to have, also maybe in the future we may want to decouple the field that is used as the target classification value (e.g. used for "training") from the field where we put the actual classification value assigned by the Classifier to the document.
          However I'm generally fine with the current state, but it'd be good if we could fix the ant idea and ant eclipse targets to correctly set Lucene Classification jar into Solr's classpath otherwise no one will be able to e.g. run tests from IDE anymore after this patch has been committed.

          Show
          teofili Tommaso Teofili added a comment - After having had a closer look at the code I think it generally looks good. One thing I noticed is that the Query parameter to build Classifiers  that use a subset of the index is missing, while it's ok for a first implementation I think that may be good to have, also maybe in the future we may want to decouple the field that is used as the target classification value (e.g. used for "training") from the field where we put the actual classification value assigned by the Classifier to the document. However I'm generally fine with the current state, but it'd be good if we could fix the ant idea and ant eclipse targets to correctly set Lucene Classification jar into Solr's classpath otherwise no one will be able to e.g. run tests from IDE anymore after this patch has been committed.
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          Thanks Tommaso ! Only one clarification, in the patch you attached I see :

          + jar-misc, jar-spatial, jar-grouping, jar-queries, jar-queryparser, jar-join, jar-sandbox, jar-classification">

          As the only difference, as the other 2 lines modified seem to be already in the original patch.
          Am I right ?
          Thanks anyway to point me that

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - Thanks Tommaso ! Only one clarification, in the patch you attached I see : + jar-misc, jar-spatial, jar-grouping, jar-queries, jar-queryparser, jar-join, jar-sandbox, jar-classification"> As the only difference, as the other 2 lines modified seem to be already in the original patch. Am I right ? Thanks anyway to point me that
          Hide
          teofili Tommaso Teofili added a comment -

          yes Alessandro Benedetti, that's what I had to add in order to make it possible for solr-core to compile correctly.

          Show
          teofili Tommaso Teofili added a comment - yes Alessandro Benedetti , that's what I had to add in order to make it possible for solr-core to compile correctly.
          Hide
          teofili Tommaso Teofili added a comment -

          Alessandro Benedetti is there any chance you can provide a new version of the patch that fixes the classpaths for ant idea and ant eclipse?

          Show
          teofili Tommaso Teofili added a comment - Alessandro Benedetti is there any chance you can provide a new version of the patch that fixes the classpaths for ant idea and ant eclipse?
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          Tommaso Teofili I ws on Holidays, I take a look when I am back , in a couple of days !
          Thanks !

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - Tommaso Teofili I ws on Holidays, I take a look when I am back , in a couple of days ! Thanks !
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          ant idea has been fixed, eclipse should work.
          Any feedback from eclipse users ?
          I run ant build and it seems ok as well.
          I agree 100% with your suggested improvements Tommaso Teofili , but I think we can add them later !

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - ant idea has been fixed, eclipse should work. Any feedback from eclipse users ? I run ant build and it seems ok as well. I agree 100% with your suggested improvements Tommaso Teofili , but I think we can add them later !
          Hide
          teofili Tommaso Teofili added a comment -

          Any feedback from eclipse users ?

          Alessandro Benedetti did you try yourself with eclipse ?

          Show
          teofili Tommaso Teofili added a comment - Any feedback from eclipse users ? Alessandro Benedetti did you try yourself with eclipse ?
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          Hi Tommaso,
          I didn't because I don't have Eclipse installed at the minute, and I don't use it since few years ago!

          My assumption that it should work is because I took a look to :

          dev-tools/eclipse/dot.classpath.xsl

          And it seems to not contain specific libraries like the IntelliJ Idea does.
          Any Eclipse User feedback could be beneficial.
          If we get nothing, I will eventually install Eclipse on my own and try!

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - Hi Tommaso, I didn't because I don't have Eclipse installed at the minute, and I don't use it since few years ago! My assumption that it should work is because I took a look to : dev-tools/eclipse/dot.classpath.xsl And it seems to not contain specific libraries like the IntelliJ Idea does. Any Eclipse User feedback could be beneficial. If we get nothing, I will eventually install Eclipse on my own and try!
          Hide
          teofili Tommaso Teofili added a comment - - edited

          Alessandro, I also use IntelliJ, however I wanted to make sure that we do not break other devs' setups just because they use a different IDE.
          Usually this is not needed but since we observed that we needed to adjust the classpaths for IntelliJ to work it was mandatory to check that also Eclipse setup doesn't get screwed up.
          Anyway, installing and checking with ant eclipse was a ~10 minutes thing:

          • download eclipse 5m
          • ant eclipse 1m
          • import lucene-solr into eclipse 1m
          • check setup is clean (e.g. check code compiles, run a test, etc.) 1m
          • close and delete eclipse 1m
            Since everything seemed to work I'll proceed with committing this patch.
          Show
          teofili Tommaso Teofili added a comment - - edited Alessandro, I also use IntelliJ, however I wanted to make sure that we do not break other devs' setups just because they use a different IDE. Usually this is not needed but since we observed that we needed to adjust the classpaths for IntelliJ to work it was mandatory to check that also Eclipse setup doesn't get screwed up. Anyway, installing and checking with ant eclipse was a ~10 minutes thing: download eclipse 5m ant eclipse 1m import lucene-solr into eclipse 1m check setup is clean (e.g. check code compiles, run a test, etc.) 1m close and delete eclipse 1m Since everything seemed to work I'll proceed with committing this patch.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 5801caab6c5fb881a590e06401096ea04111d905 in lucene-solr's branch refs/heads/master from Tommaso Teofili
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5801caa ]

          SOLR-7739 - applied patch from Alessandro Benedetti for integrating Lucene classification into Solr

          Show
          jira-bot ASF subversion and git services added a comment - Commit 5801caab6c5fb881a590e06401096ea04111d905 in lucene-solr's branch refs/heads/master from Tommaso Teofili [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5801caa ] SOLR-7739 - applied patch from Alessandro Benedetti for integrating Lucene classification into Solr
          Hide
          teofili Tommaso Teofili added a comment -

          Alessandro Benedetti would it be possible for you to create a documentation page on our wiki ?

          Show
          teofili Tommaso Teofili added a comment - Alessandro Benedetti would it be possible for you to create a documentation page on our wiki ?
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          Absolutely, As soon as I have time I will do that, is necessary to obtain the permission in some way ?
          I will check later with Cassandra ( she's the responsible if I remember well) or in the mailing list.
          Thanks !

          Cheers

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - Absolutely, As soon as I have time I will do that, is necessary to obtain the permission in some way ? I will check later with Cassandra ( she's the responsible if I remember well) or in the mailing list. Thanks ! Cheers
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 15fdfd3caa54c1e6d8cf3435ded285c3b65bea8c in lucene-solr's branch refs/heads/branch_6x from Tommaso Teofili
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=15fdfd3 ]

          SOLR-7739 - applied patch from Alessandro Benedetti for integrating Lucene classification into Solr
          (cherry picked from commit 5801caa)

          Show
          jira-bot ASF subversion and git services added a comment - Commit 15fdfd3caa54c1e6d8cf3435ded285c3b65bea8c in lucene-solr's branch refs/heads/branch_6x from Tommaso Teofili [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=15fdfd3 ] SOLR-7739 - applied patch from Alessandro Benedetti for integrating Lucene classification into Solr (cherry picked from commit 5801caa)
          Hide
          hossman Hoss Man added a comment -

          Manually correcting fixVersion per Step #S5 of LUCENE-7271

          Show
          hossman Hoss Man added a comment - Manually correcting fixVersion per Step #S5 of LUCENE-7271
          Hide
          tomas0413 Tomas Ramanauskas added a comment -

          Hello, I tried this feature few days ago, but I couldn't get it to work. I left couple of comments on this Blog site with my examples:

          https://alexbenedetti.blogspot.co.uk/2015/07/solr-document-classification-part-1.html

          May someone who was able to get this feature to work, share some query examples?

          Show
          tomas0413 Tomas Ramanauskas added a comment - Hello, I tried this feature few days ago, but I couldn't get it to work. I left couple of comments on this Blog site with my examples: https://alexbenedetti.blogspot.co.uk/2015/07/solr-document-classification-part-1.html May someone who was able to get this feature to work, share some query examples?
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          Hi Tomas, the best place for this questions I would say it is the Solr mailing list.
          I would suggest you to write an email with your :

          1) Solr Version
          2) Solr Configuration
          3) Solr Index you built

          And then I can give you an extensive response

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - Hi Tomas, the best place for this questions I would say it is the Solr mailing list. I would suggest you to write an email with your : 1) Solr Version 2) Solr Configuration 3) Solr Index you built And then I can give you an extensive response
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          Gents,
          is this included in the 6.1 release Adrien Grand Hoss Man ?
          Because from the code perspective and the "Fix Version" Jira field, I can say yes, but it is not in the release notes ...
          Am I missing anything ?

          Cheers

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - Gents, is this included in the 6.1 release Adrien Grand Hoss Man ? Because from the code perspective and the "Fix Version" Jira field, I can say yes, but it is not in the release notes ... Am I missing anything ? Cheers
          Hide
          tomas0413 Tomas Ramanauskas added a comment -

          >> Solr Version:
          6.1.0-68

          >> Solr Configuration

          I will attach solrconfig.xml

          >> I build in index using this command:
          ./solr create -c demo

          What else shall I provide?

          Show
          tomas0413 Tomas Ramanauskas added a comment - >> Solr Version: 6.1.0-68 >> Solr Configuration I will attach solrconfig.xml >> I build in index using this command: ./solr create -c demo What else shall I provide?
          Hide
          tomas0413 Tomas Ramanauskas added a comment -

          Not sure, I can attach the file here.

          This is the configuration file:
          ./server/solr/demo/conf/solrconfig.xml

          <?xml version="1.0" encoding="UTF-8" ?>
          <!--
           Licensed to the Apache Software Foundation (ASF) under one or more
           contributor license agreements.  See the NOTICE file distributed with
           this work for additional information regarding copyright ownership.
           The ASF licenses this file to You under the Apache License, Version 2.0
           (the "License"); you may not use this file except in compliance with
           the License.  You may obtain a copy of the License at
          
               http://www.apache.org/licenses/LICENSE-2.0
          
           Unless required by applicable law or agreed to in writing, software
           distributed under the License is distributed on an "AS IS" BASIS,
           WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
           See the License for the specific language governing permissions and
           limitations under the License.
          -->
          
          <!-- 
               For more details about configurations options that may appear in
               this file, see http://wiki.apache.org/solr/SolrConfigXml. 
          -->
          <config>
            <!-- In all configuration below, a prefix of "solr." for class names
                 is an alias that causes solr to search appropriate packages,
                 including org.apache.solr.(search|update|request|core|analysis)
          
                 You may also specify a fully qualified Java classname if you
                 have your own custom plugins.
              -->
          
            <!-- Controls what version of Lucene various components of Solr
                 adhere to.  Generally, you want to use the latest version to
                 get all bug fixes and improvements. It is highly recommended
                 that you fully re-index after changing this setting as it can
                 affect both how text is indexed and queried.
            -->
            <luceneMatchVersion>6.1.0</luceneMatchVersion>
          
            <!-- <lib/> directives can be used to instruct Solr to load any Jars
                 identified and use them to resolve any "plugins" specified in
                 your solrconfig.xml or schema.xml (ie: Analyzers, Request
                 Handlers, etc...).
          
                 All directories and paths are resolved relative to the
                 instanceDir.
          
                 Please note that <lib/> directives are processed in the order
                 that they appear in your solrconfig.xml file, and are "stacked" 
                 on top of each other when building a ClassLoader - so if you have 
                 plugin jars with dependencies on other jars, the "lower level" 
                 dependency jars should be loaded first.
          
                 If a "./lib" directory exists in your instanceDir, all files
                 found in it are included as if you had used the following
                 syntax...
                 
                        <lib dir="./lib" />
              -->
          
            <!-- A 'dir' option by itself adds any files found in the directory 
                 to the classpath, this is useful for including all jars in a
                 directory.
          
                 When a 'regex' is specified in addition to a 'dir', only the
                 files in that directory which completely match the regex
                 (anchored on both ends) will be included.
          
                 If a 'dir' option (with or without a regex) is used and nothing
                 is found that matches, a warning will be logged.
          
                 The examples below can be used to load some solr-contribs along 
                 with their external dependencies.
              -->
            <lib dir="${solr.install.dir:../../../..}/contrib/extraction/lib" regex=".*\.jar" />
            <lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-cell-\d.*\.jar" />
          
            <lib dir="${solr.install.dir:../../../..}/contrib/clustering/lib/" regex=".*\.jar" />
            <lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-clustering-\d.*\.jar" />
          
            <lib dir="${solr.install.dir:../../../..}/contrib/langid/lib/" regex=".*\.jar" />
            <lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-langid-\d.*\.jar" />
          
            <lib dir="${solr.install.dir:../../../..}/contrib/velocity/lib" regex=".*\.jar" />
            <lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-velocity-\d.*\.jar" />
            <!-- an exact 'path' can be used instead of a 'dir' to specify a 
                 specific jar file.  This will cause a serious error to be logged 
                 if it can't be loaded.
              -->
            <!--
               <lib path="../a-jar-that-does-not-exist.jar" /> 
            -->
          
            <!-- Data Directory
          
                 Used to specify an alternate directory to hold all index data
                 other than the default ./data under the Solr home.  If
                 replication is in use, this should match the replication
                 configuration.
              -->
            <dataDir>${solr.data.dir:}</dataDir>
          
          
            <!-- The DirectoryFactory to use for indexes.
                 
                 solr.StandardDirectoryFactory is filesystem
                 based and tries to pick the best implementation for the current
                 JVM and platform.  solr.NRTCachingDirectoryFactory, the default,
                 wraps solr.StandardDirectoryFactory and caches small files in memory
                 for better NRT performance.
          
                 One can force a particular implementation via solr.MMapDirectoryFactory,
                 solr.NIOFSDirectoryFactory, or solr.SimpleFSDirectoryFactory.
          
                 solr.RAMDirectoryFactory is memory based, not
                 persistent, and doesn't work with replication.
              -->
            <directoryFactory name="DirectoryFactory"
                              class="${solr.directoryFactory:solr.NRTCachingDirectoryFactory}"/>
          
            <!-- The CodecFactory for defining the format of the inverted index.
                 The default implementation is SchemaCodecFactory, which is the official Lucene
                 index format, but hooks into the schema to provide per-field customization of
                 the postings lists and per-document values in the fieldType element
                 (postingsFormat/docValuesFormat). Note that most of the alternative implementations
                 are experimental, so if you choose to customize the index format, it's a good
                 idea to convert back to the official format e.g. via IndexWriter.addIndexes(IndexReader)
                 before upgrading to a newer version to avoid unnecessary reindexing.
                 A "compressionMode" string element can be added to <codecFactory> to choose 
                 between the existing compression modes in the default codec: "BEST_SPEED" (default)
                 or "BEST_COMPRESSION".
            -->
            <codecFactory class="solr.SchemaCodecFactory"/>
          
            <!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                 Index Config - These settings control low-level behavior of indexing
                 Most example settings here show the default value, but are commented
                 out, to more easily see where customizations have been made.
                 
                 Note: This replaces <indexDefaults> and <mainIndex> from older versions
                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->
            <indexConfig>
              <!-- maxFieldLength was removed in 4.0. To get similar behavior, include a 
                   LimitTokenCountFilterFactory in your fieldType definition. E.g. 
               <filter class="solr.LimitTokenCountFilterFactory" maxTokenCount="10000"/>
              -->
              <!-- Maximum time to wait for a write lock (ms) for an IndexWriter. Default: 1000 -->
              <!-- <writeLockTimeout>1000</writeLockTimeout>  -->
          
              <!-- Expert: Enabling compound file will use less files for the index, 
                   using fewer file descriptors on the expense of performance decrease. 
                   Default in Lucene is "true". Default in Solr is "false" (since 3.6) -->
              <!-- <useCompoundFile>false</useCompoundFile> -->
          
              <!-- ramBufferSizeMB sets the amount of RAM that may be used by Lucene
                   indexing for buffering added documents and deletions before they are
                   flushed to the Directory.
                   maxBufferedDocs sets a limit on the number of documents buffered
                   before flushing.
                   If both ramBufferSizeMB and maxBufferedDocs is set, then
                   Lucene will flush based on whichever limit is hit first.  -->
              <!-- <ramBufferSizeMB>100</ramBufferSizeMB> -->
              <!-- <maxBufferedDocs>1000</maxBufferedDocs> -->
          
              <!-- Expert: Merge Policy 
                   The Merge Policy in Lucene controls how merging of segments is done.
                   The default since Solr/Lucene 3.3 is TieredMergePolicy.
                   The default since Lucene 2.3 was the LogByteSizeMergePolicy,
                   Even older versions of Lucene used LogDocMergePolicy.
                -->
              <!--
                  <mergePolicyFactory class="org.apache.solr.index.TieredMergePolicyFactory">
                    <int name="maxMergeAtOnce">10</int>
                    <int name="segmentsPerTier">10</int>
                    <double name="noCFSRatio">0.1</double>
                  </mergePolicyFactory>
                -->
          
              <!-- Expert: Merge Scheduler
                   The Merge Scheduler in Lucene controls how merges are
                   performed.  The ConcurrentMergeScheduler (Lucene 2.3 default)
                   can perform merges in the background using separate threads.
                   The SerialMergeScheduler (Lucene 2.2 default) does not.
               -->
              <!-- 
                 <mergeScheduler class="org.apache.lucene.index.ConcurrentMergeScheduler"/>
                 -->
          
              <!-- LockFactory 
          
                   This option specifies which Lucene LockFactory implementation
                   to use.
                
                   single = SingleInstanceLockFactory - suggested for a
                            read-only index or when there is no possibility of
                            another process trying to modify the index.
                   native = NativeFSLockFactory - uses OS native file locking.
                            Do not use when multiple solr webapps in the same
                            JVM are attempting to share a single index.
                   simple = SimpleFSLockFactory  - uses a plain file for locking
          
                   Defaults: 'native' is default for Solr3.6 and later, otherwise
                             'simple' is the default
          
                   More details on the nuances of each LockFactory...
                   http://wiki.apache.org/lucene-java/AvailableLockFactories
              -->
              <lockType>${solr.lock.type:native}</lockType>
          
              <!-- Commit Deletion Policy
                   Custom deletion policies can be specified here. The class must
                   implement org.apache.lucene.index.IndexDeletionPolicy.
          
                   The default Solr IndexDeletionPolicy implementation supports
                   deleting index commit points on number of commits, age of
                   commit point and optimized status.
                   
                   The latest commit point should always be preserved regardless
                   of the criteria.
              -->
              <!-- 
              <deletionPolicy class="solr.SolrDeletionPolicy">
              -->
              <!-- The number of commit points to be kept -->
              <!-- <str name="maxCommitsToKeep">1</str> -->
              <!-- The number of optimized commit points to be kept -->
              <!-- <str name="maxOptimizedCommitsToKeep">0</str> -->
              <!--
                  Delete all commit points once they have reached the given age.
                  Supports DateMathParser syntax e.g.
                -->
              <!--
                 <str name="maxCommitAge">30MINUTES</str>
                 <str name="maxCommitAge">1DAY</str>
              -->
              <!-- 
              </deletionPolicy>
              -->
          
              <!-- Lucene Infostream
                 
                   To aid in advanced debugging, Lucene provides an "InfoStream"
                   of detailed information when indexing.
          
                   Setting The value to true will instruct the underlying Lucene
                   IndexWriter to write its debugging info the specified file
                -->
              <!-- <infoStream file="INFOSTREAM.txt">false</infoStream> -->
            </indexConfig>
          
          
            <!-- JMX
                 
                 This example enables JMX if and only if an existing MBeanServer
                 is found, use this if you want to configure JMX through JVM
                 parameters. Remove this to disable exposing Solr configuration
                 and statistics to JMX.
          
                 For more details see http://wiki.apache.org/solr/SolrJmx
              -->
            <jmx />
            <!-- If you want to connect to a particular server, specify the
                 agentId 
              -->
            <!-- <jmx agentId="myAgent" /> -->
            <!-- If you want to start a new MBeanServer, specify the serviceUrl -->
            <!-- <jmx serviceUrl="service:jmx:rmi:///jndi/rmi://localhost:9999/solr"/>
              -->
          
            <!-- The default high-performance update handler -->
            <updateHandler class="solr.DirectUpdateHandler2">
          
              <!-- Enables a transaction log, used for real-time get, durability, and
                   and solr cloud replica recovery.  The log can grow as big as
                   uncommitted changes to the index, so use of a hard autoCommit
                   is recommended (see below).
                   "dir" - the target directory for transaction logs, defaults to the
                          solr data directory.
                   "numVersionBuckets" - sets the number of buckets used to keep
                          track of max version values when checking for re-ordered
                          updates; increase this value to reduce the cost of
                          synchronizing access to version buckets during high-volume
                          indexing, this requires 8 bytes (long) * numVersionBuckets
                          of heap space per Solr core.
              -->
              <updateLog>
                <str name="dir">${solr.ulog.dir:}</str>
                <int name="numVersionBuckets">${solr.ulog.numVersionBuckets:65536}</int>
              </updateLog>
          
              <!-- AutoCommit
          
                   Perform a hard commit automatically under certain conditions.
                   Instead of enabling autoCommit, consider using "commitWithin"
                   when adding documents. 
          
                   http://wiki.apache.org/solr/UpdateXmlMessages
          
                   maxDocs - Maximum number of documents to add since the last
                             commit before automatically triggering a new commit.
          
                   maxTime - Maximum amount of time in ms that is allowed to pass
                             since a document was added before automatically
                             triggering a new commit. 
                   openSearcher - if false, the commit causes recent index changes
                     to be flushed to stable storage, but does not cause a new
                     searcher to be opened to make those changes visible.
          
                   If the updateLog is enabled, then it's highly recommended to
                   have some sort of hard autoCommit to limit the log size.
                -->
              <autoCommit>
                <maxTime>${solr.autoCommit.maxTime:15000}</maxTime>
                <openSearcher>false</openSearcher>
              </autoCommit>
          
              <!-- softAutoCommit is like autoCommit except it causes a
                   'soft' commit which only ensures that changes are visible
                   but does not ensure that data is synced to disk.  This is
                   faster and more near-realtime friendly than a hard commit.
                -->
          
              <autoSoftCommit>
                <maxTime>${solr.autoSoftCommit.maxTime:-1}</maxTime>
              </autoSoftCommit>
          
              <!-- Update Related Event Listeners
                   
                   Various IndexWriter related events can trigger Listeners to
                   take actions.
          
                   postCommit - fired after every commit or optimize command
                   postOptimize - fired after every optimize command
                -->
              <!-- The RunExecutableListener executes an external command from a
                   hook such as postCommit or postOptimize.
                   
                   exe - the name of the executable to run
                   dir - dir to use as the current working directory. (default=".")
                   wait - the calling thread waits until the executable returns. 
                          (default="true")
                   args - the arguments to pass to the program.  (default is none)
                   env - environment variables to set.  (default is none)
                -->
              <!-- This example shows how RunExecutableListener could be used
                   with the script based replication...
                   http://wiki.apache.org/solr/CollectionDistribution
                -->
              <!--
                 <listener event="postCommit" class="solr.RunExecutableListener">
                   <str name="exe">solr/bin/snapshooter</str>
                   <str name="dir">.</str>
                   <bool name="wait">true</bool>
                   <arr name="args"> <str>arg1</str> <str>arg2</str> </arr>
                   <arr name="env"> <str>MYVAR=val1</str> </arr>
                 </listener>
                -->
          
            </updateHandler>
          
            <!-- IndexReaderFactory
          
                 Use the following format to specify a custom IndexReaderFactory,
                 which allows for alternate IndexReader implementations.
          
                 ** Experimental Feature **
          
                 Please note - Using a custom IndexReaderFactory may prevent
                 certain other features from working. The API to
                 IndexReaderFactory may change without warning or may even be
                 removed from future releases if the problems cannot be
                 resolved.
          
          
                 ** Features that may not work with custom IndexReaderFactory **
          
                 The ReplicationHandler assumes a disk-resident index. Using a
                 custom IndexReader implementation may cause incompatibility
                 with ReplicationHandler and may cause replication to not work
                 correctly. See SOLR-1366 for details.
          
              -->
            <!--
            <indexReaderFactory name="IndexReaderFactory" class="package.class">
              <str name="someArg">Some Value</str>
            </indexReaderFactory >
            -->
          
            <!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                 Query section - these settings control query time things like caches
                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->
            <query>
              <!-- Max Boolean Clauses
          
                   Maximum number of clauses in each BooleanQuery,  an exception
                   is thrown if exceeded.
          
                   ** WARNING **
                   
                   This option actually modifies a global Lucene property that
                   will affect all SolrCores.  If multiple solrconfig.xml files
                   disagree on this property, the value at any given moment will
                   be based on the last SolrCore to be initialized.
                   
                -->
              <maxBooleanClauses>1024</maxBooleanClauses>
          
          
              <!-- Solr Internal Query Caches
          
                   There are two implementations of cache available for Solr,
                   LRUCache, based on a synchronized LinkedHashMap, and
                   FastLRUCache, based on a ConcurrentHashMap.  
          
                   FastLRUCache has faster gets and slower puts in single
                   threaded operation and thus is generally faster than LRUCache
                   when the hit ratio of the cache is high (> 75%), and may be
                   faster under other scenarios on multi-cpu systems.
              -->
          
              <!-- Filter Cache
          
                   Cache used by SolrIndexSearcher for filters (DocSets),
                   unordered sets of *all* documents that match a query.  When a
                   new searcher is opened, its caches may be prepopulated or
                   "autowarmed" using data from caches in the old searcher.
                   autowarmCount is the number of items to prepopulate.  For
                   LRUCache, the autowarmed items will be the most recently
                   accessed items.
          
                   Parameters:
                     class - the SolrCache implementation LRUCache or
                         (LRUCache or FastLRUCache)
                     size - the maximum number of entries in the cache
                     initialSize - the initial capacity (number of entries) of
                         the cache.  (see java.util.HashMap)
                     autowarmCount - the number of entries to prepopulate from
                         and old cache.
                -->
              <filterCache class="solr.FastLRUCache"
                           size="512"
                           initialSize="512"
                           autowarmCount="0"/>
          
              <!-- Query Result Cache
                   
                   Caches results of searches - ordered lists of document ids
                   (DocList) based on a query, a sort, and the range of documents requested.
                   Additional supported parameter by LRUCache:
                      maxRamMB - the maximum amount of RAM (in MB) that this cache is allowed
                                 to occupy
                -->
              <queryResultCache class="solr.LRUCache"
                                size="512"
                                initialSize="512"
                                autowarmCount="0"/>
          
              <!-- Document Cache
          
                   Caches Lucene Document objects (the stored fields for each
                   document).  Since Lucene internal document ids are transient,
                   this cache will not be autowarmed.
                -->
              <documentCache class="solr.LRUCache"
                             size="512"
                             initialSize="512"
                             autowarmCount="0"/>
          
              <!-- custom cache currently used by block join -->
              <cache name="perSegFilter"
                     class="solr.search.LRUCache"
                     size="10"
                     initialSize="0"
                     autowarmCount="10"
                     regenerator="solr.NoOpRegenerator" />
          
              <!-- Field Value Cache
                   
                   Cache used to hold field values that are quickly accessible
                   by document id.  The fieldValueCache is created by default
                   even if not configured here.
                -->
              <!--
                 <fieldValueCache class="solr.FastLRUCache"
                                  size="512"
                                  autowarmCount="128"
                                  showItems="32" />
                -->
          
              <!-- Custom Cache
          
                   Example of a generic cache.  These caches may be accessed by
                   name through SolrIndexSearcher.getCache(),cacheLookup(), and
                   cacheInsert().  The purpose is to enable easy caching of
                   user/application level data.  The regenerator argument should
                   be specified as an implementation of solr.CacheRegenerator 
                   if autowarming is desired.  
                -->
              <!--
                 <cache name="myUserCache"
                        class="solr.LRUCache"
                        size="4096"
                        initialSize="1024"
                        autowarmCount="1024"
                        regenerator="com.mycompany.MyRegenerator"
                        />
                -->
          
          
              <!-- Lazy Field Loading
          
                   If true, stored fields that are not requested will be loaded
                   lazily.  This can result in a significant speed improvement
                   if the usual case is to not load all stored fields,
                   especially if the skipped fields are large compressed text
                   fields.
              -->
              <enableLazyFieldLoading>true</enableLazyFieldLoading>
          
              <!-- Use Filter For Sorted Query
           
                   A possible optimization that attempts to use a filter to
                   satisfy a search.  If the requested sort does not include
                   score, then the filterCache will be checked for a filter
                   matching the query. If found, the filter will be used as the
                   source of document ids, and then the sort will be applied to
                   that.
           
                   For most situations, this will not be useful unless you
                   frequently get the same search repeatedly with different sort
                   options, and none of them ever use "score"
                -->
              <!--
                 <useFilterForSortedQuery>true</useFilterForSortedQuery>
                -->
          
              <!-- Result Window Size
           
                   An optimization for use with the queryResultCache.  When a search
                   is requested, a superset of the requested number of document ids
                   are collected.  For example, if a search for a particular query
                   requests matching documents 10 through 19, and queryWindowSize is 50,
                   then documents 0 through 49 will be collected and cached.  Any further
                   requests in that range can be satisfied via the cache.  
                -->
              <queryResultWindowSize>20</queryResultWindowSize>
          
              <!-- Maximum number of documents to cache for any entry in the
                   queryResultCache. 
                -->
              <queryResultMaxDocsCached>200</queryResultMaxDocsCached>
          
              <!-- Query Related Event Listeners
           
                   Various IndexSearcher related events can trigger Listeners to
                   take actions.
           
                   newSearcher - fired whenever a new searcher is being prepared
                   and there is a current searcher handling requests (aka
                   registered).  It can be used to prime certain caches to
                   prevent long request times for certain requests.
           
                   firstSearcher - fired whenever a new searcher is being
                   prepared but there is no current registered searcher to handle
                   requests or to gain autowarming data from.
           
                   
                -->
              <!-- QuerySenderListener takes an array of NamedList and executes a
                   local query request for each NamedList in sequence. 
                -->
              <listener event="newSearcher" class="solr.QuerySenderListener">
                <arr name="queries">
                  <!--
                     <lst><str name="q">solr</str><str name="sort">price asc</str></lst>
                     <lst><str name="q">rocks</str><str name="sort">weight asc</str></lst>
                    -->
                </arr>
              </listener>
              <listener event="firstSearcher" class="solr.QuerySenderListener">
                <arr name="queries">
                  <!--
                  <lst>
                    <str name="q">static firstSearcher warming in solrconfig.xml</str>
                  </lst>
                  -->
                </arr>
              </listener>
          
              <!-- Use Cold Searcher
          
                   If a search request comes in and there is no current
                   registered searcher, then immediately register the still
                   warming searcher and use it.  If "false" then all requests
                   will block until the first searcher is done warming.
                -->
              <useColdSearcher>false</useColdSearcher>
          
              <!-- Max Warming Searchers
                   
                   Maximum number of searchers that may be warming in the
                   background concurrently.  An error is returned if this limit
                   is exceeded.
          
                   Recommend values of 1-2 for read-only slaves, higher for
                   masters w/o cache warming.
                -->
              <maxWarmingSearchers>2</maxWarmingSearchers>
          
            </query>
          
          
            <!-- Request Dispatcher
          
                 This section contains instructions for how the SolrDispatchFilter
                 should behave when processing requests for this SolrCore.
          
                 handleSelect is a legacy option that affects the behavior of requests
                 such as /select?qt=XXX
          
                 handleSelect="true" will cause the SolrDispatchFilter to process
                 the request and dispatch the query to a handler specified by the 
                 "qt" param, assuming "/select" isn't already registered.
          
                 handleSelect="false" will cause the SolrDispatchFilter to
                 ignore "/select" requests, resulting in a 404 unless a handler
                 is explicitly registered with the name "/select"
          
                 handleSelect="true" is not recommended for new users, but is the default
                 for backwards compatibility
              -->
            <requestDispatcher handleSelect="false" >
              <!-- Request Parsing
          
                   These settings indicate how Solr Requests may be parsed, and
                   what restrictions may be placed on the ContentStreams from
                   those requests
          
                   enableRemoteStreaming - enables use of the stream.file
                   and stream.url parameters for specifying remote streams.
          
                   multipartUploadLimitInKB - specifies the max size (in KiB) of
                   Multipart File Uploads that Solr will allow in a Request.
                   
                   formdataUploadLimitInKB - specifies the max size (in KiB) of
                   form data (application/x-www-form-urlencoded) sent via
                   POST. You can use POST to pass request parameters not
                   fitting into the URL.
                   
                   addHttpRequestToContext - if set to true, it will instruct
                   the requestParsers to include the original HttpServletRequest
                   object in the context map of the SolrQueryRequest under the 
                   key "httpRequest". It will not be used by any of the existing
                   Solr components, but may be useful when developing custom 
                   plugins.
                   
                   *** WARNING ***
                   The settings below authorize Solr to fetch remote files, You
                   should make sure your system has some authentication before
                   using enableRemoteStreaming="true"
          
                -->
              <requestParsers enableRemoteStreaming="true"
                              multipartUploadLimitInKB="2048000"
                              formdataUploadLimitInKB="2048"
                              addHttpRequestToContext="false"/>
          
              <!-- HTTP Caching
          
                   Set HTTP caching related parameters (for proxy caches and clients).
          
                   The options below instruct Solr not to output any HTTP Caching
                   related headers
                -->
              <httpCaching never304="true" />
              <!-- If you include a <cacheControl> directive, it will be used to
                   generate a Cache-Control header (as well as an Expires header
                   if the value contains "max-age=")
                   
                   By default, no Cache-Control header is generated.
                   
                   You can use the <cacheControl> option even if you have set
                   never304="true"
                -->
              <!--
                 <httpCaching never304="true" >
                   <cacheControl>max-age=30, public</cacheControl> 
                 </httpCaching>
                -->
              <!-- To enable Solr to respond with automatically generated HTTP
                   Caching headers, and to response to Cache Validation requests
                   correctly, set the value of never304="false"
                   
                   This will cause Solr to generate Last-Modified and ETag
                   headers based on the properties of the Index.
          
                   The following options can also be specified to affect the
                   values of these headers...
          
                   lastModFrom - the default value is "openTime" which means the
                   Last-Modified value (and validation against If-Modified-Since
                   requests) will all be relative to when the current Searcher
                   was opened.  You can change it to lastModFrom="dirLastMod" if
                   you want the value to exactly correspond to when the physical
                   index was last modified.
          
                   etagSeed="..." is an option you can change to force the ETag
                   header (and validation against If-None-Match requests) to be
                   different even if the index has not changed (ie: when making
                   significant changes to your config file)
          
                   (lastModifiedFrom and etagSeed are both ignored if you use
                   the never304="true" option)
                -->
              <!--
                 <httpCaching lastModifiedFrom="openTime"
                              etagSeed="Solr">
                   <cacheControl>max-age=30, public</cacheControl> 
                 </httpCaching>
                -->
            </requestDispatcher>
          
            <!-- Request Handlers 
          
                 http://wiki.apache.org/solr/SolrRequestHandler
          
                 Incoming queries will be dispatched to a specific handler by name
                 based on the path specified in the request.
          
                 Legacy behavior: If the request path uses "/select" but no Request
                 Handler has that name, and if handleSelect="true" has been specified in
                 the requestDispatcher, then the Request Handler is dispatched based on
                 the qt parameter.  Handlers without a leading '/' are accessed this way
                 like so: http://host/app/[core/]select?qt=name  If no qt is
                 given, then the requestHandler that declares default="true" will be
                 used or the one named "standard".
          
                 If a Request Handler is declared with startup="lazy", then it will
                 not be initialized until the first request that uses it.
          
              -->
            <!-- SearchHandler
          
                 http://wiki.apache.org/solr/SearchHandler
          
                 For processing Search Queries, the primary Request Handler
                 provided with Solr is "SearchHandler" It delegates to a sequent
                 of SearchComponents (see below) and supports distributed
                 queries across multiple shards
              -->
            <requestHandler name="/select" class="solr.SearchHandler">
              <!-- default values for query parameters can be specified, these
                   will be overridden by parameters in the request
                -->
              <lst name="defaults">
                <str name="echoParams">explicit</str>
                <int name="rows">10</int>
                <!-- <str name="df">text</str> -->
              </lst>
              <!-- In addition to defaults, "appends" params can be specified
                   to identify values which should be appended to the list of
                   multi-val params from the query (or the existing "defaults").
                -->
              <!-- In this example, the param "fq=instock:true" would be appended to
                   any query time fq params the user may specify, as a mechanism for
                   partitioning the index, independent of any user selected filtering
                   that may also be desired (perhaps as a result of faceted searching).
          
                   NOTE: there is *absolutely* nothing a client can do to prevent these
                   "appends" values from being used, so don't use this mechanism
                   unless you are sure you always want it.
                -->
              <!--
                 <lst name="appends">
                   <str name="fq">inStock:true</str>
                 </lst>
                -->
              <!-- "invariants" are a way of letting the Solr maintainer lock down
                   the options available to Solr clients.  Any params values
                   specified here are used regardless of what values may be specified
                   in either the query, the "defaults", or the "appends" params.
          
                   In this example, the facet.field and facet.query params would
                   be fixed, limiting the facets clients can use.  Faceting is
                   not turned on by default - but if the client does specify
                   facet=true in the request, these are the only facets they
                   will be able to see counts for; regardless of what other
                   facet.field or facet.query params they may specify.
          
                   NOTE: there is *absolutely* nothing a client can do to prevent these
                   "invariants" values from being used, so don't use this mechanism
                   unless you are sure you always want it.
                -->
              <!--
                 <lst name="invariants">
                   <str name="facet.field">cat</str>
                   <str name="facet.field">manu_exact</str>
                   <str name="facet.query">price:[* TO 500]</str>
                   <str name="facet.query">price:[500 TO *]</str>
                 </lst>
                -->
              <!-- If the default list of SearchComponents is not desired, that
                   list can either be overridden completely, or components can be
                   prepended or appended to the default list.  (see below)
                -->
              <!--
                 <arr name="components">
                   <str>nameOfCustomComponent1</str>
                   <str>nameOfCustomComponent2</str>
                 </arr>
                -->
            </requestHandler>
          
            <!-- A request handler that returns indented JSON by default -->
            <requestHandler name="/query" class="solr.SearchHandler">
              <lst name="defaults">
                <str name="echoParams">explicit</str>
                <str name="wt">json</str>
                <str name="indent">true</str>
              </lst>
            </requestHandler>
          
          
            <!-- A Robust Example
                 
                 This example SearchHandler declaration shows off usage of the
                 SearchHandler with many defaults declared
          
                 Note that multiple instances of the same Request Handler
                 (SearchHandler) can be registered multiple times with different
                 names (and different init parameters)
              -->
            <requestHandler name="/browse" class="solr.SearchHandler" useParams="query,facets,velocity,browse">
              <lst name="defaults">
                <str name="echoParams">explicit</str>
              </lst>
            </requestHandler>
          
            <initParams path="/update/**,/query,/select,/tvrh,/elevate,/spell,/browse">
              <lst name="defaults">
                <str name="df">_text_</str>
              </lst>
            </initParams>
          
            <initParams path="/update/**">
              <lst name="defaults">
                <str name="update.chain">add-unknown-fields-to-the-schema</str>
              </lst>
            </initParams>
          
            <!-- Solr Cell Update Request Handler
          
                 http://wiki.apache.org/solr/ExtractingRequestHandler 
          
              -->
            <requestHandler name="/update/extract"
                            startup="lazy"
                            class="solr.extraction.ExtractingRequestHandler" >
              <lst name="defaults">
                <str name="lowernames">true</str>
                <str name="fmap.meta">ignored_</str>
                <str name="fmap.content">_text_</str>
              </lst>
            </requestHandler>
          
            <!-- Field Analysis Request Handler
          
                 RequestHandler that provides much the same functionality as
                 analysis.jsp. Provides the ability to specify multiple field
                 types and field names in the same request and outputs
                 index-time and query-time analysis for each of them.
          
                 Request parameters are:
                 analysis.fieldname - field name whose analyzers are to be used
          
                 analysis.fieldtype - field type whose analyzers are to be used
                 analysis.fieldvalue - text for index-time analysis
                 q (or analysis.q) - text for query time analysis
                 analysis.showmatch (true|false) - When set to true and when
                     query analysis is performed, the produced tokens of the
                     field value analysis will be marked as "matched" for every
                     token that is produces by the query analysis
             -->
            <requestHandler name="/analysis/field"
                            startup="lazy"
                            class="solr.FieldAnalysisRequestHandler" />
          
          
            <!-- Document Analysis Handler
          
                 http://wiki.apache.org/solr/AnalysisRequestHandler
          
                 An analysis handler that provides a breakdown of the analysis
                 process of provided documents. This handler expects a (single)
                 content stream with the following format:
          
                 <docs>
                   <doc>
                     <field name="id">1</field>
                     <field name="name">The Name</field>
                     <field name="text">The Text Value</field>
                   </doc>
                   <doc>...</doc>
                   <doc>...</doc>
                   ...
                 </docs>
          
              Note: Each document must contain a field which serves as the
              unique key. This key is used in the returned response to associate
              an analysis breakdown to the analyzed document.
          
              Like the FieldAnalysisRequestHandler, this handler also supports
              query analysis by sending either an "analysis.query" or "q"
              request parameter that holds the query text to be analyzed. It
              also supports the "analysis.showmatch" parameter which when set to
              true, all field tokens that match the query tokens will be marked
              as a "match". 
            -->
            <requestHandler name="/analysis/document"
                            class="solr.DocumentAnalysisRequestHandler"
                            startup="lazy" />
          
            <!-- Echo the request contents back to the client -->
            <requestHandler name="/debug/dump" class="solr.DumpRequestHandler" >
              <lst name="defaults">
                <str name="echoParams">explicit</str>
                <str name="echoHandler">true</str>
              </lst>
            </requestHandler>
          
            <!-- Search Components
          
                 Search components are registered to SolrCore and used by 
                 instances of SearchHandler (which can access them by name)
                 
                 By default, the following components are available:
                 
                 <searchComponent name="query"     class="solr.QueryComponent" />
                 <searchComponent name="facet"     class="solr.FacetComponent" />
                 <searchComponent name="mlt"       class="solr.MoreLikeThisComponent" />
                 <searchComponent name="highlight" class="solr.HighlightComponent" />
                 <searchComponent name="stats"     class="solr.StatsComponent" />
                 <searchComponent name="debug"     class="solr.DebugComponent" />
             
                 Default configuration in a requestHandler would look like:
          
                 <arr name="components">
                   <str>query</str>
                   <str>facet</str>
                   <str>mlt</str>
                   <str>highlight</str>
                   <str>stats</str>
                   <str>debug</str>
                 </arr>
          
                 If you register a searchComponent to one of the standard names, 
                 that will be used instead of the default.
          
                 To insert components before or after the 'standard' components, use:
              
                 <arr name="first-components">
                   <str>myFirstComponentName</str>
                 </arr>
              
                 <arr name="last-components">
                   <str>myLastComponentName</str>
                 </arr>
          
                 NOTE: The component registered with the name "debug" will
                 always be executed after the "last-components" 
                 
               -->
          
            <!-- Spell Check
          
                 The spell check component can return a list of alternative spelling
                 suggestions.  
          
                 http://wiki.apache.org/solr/SpellCheckComponent
              -->
            <searchComponent name="spellcheck" class="solr.SpellCheckComponent">
          
              <str name="queryAnalyzerFieldType">text_general</str>
          
              <!-- Multiple "Spell Checkers" can be declared and used by this
                   component
                -->
          
              <!-- a spellchecker built from a field of the main index -->
              <lst name="spellchecker">
                <str name="name">default</str>
                <str name="field">_text_</str>
                <str name="classname">solr.DirectSolrSpellChecker</str>
                <!-- the spellcheck distance measure used, the default is the internal levenshtein -->
                <str name="distanceMeasure">internal</str>
                <!-- minimum accuracy needed to be considered a valid spellcheck suggestion -->
                <float name="accuracy">0.5</float>
                <!-- the maximum #edits we consider when enumerating terms: can be 1 or 2 -->
                <int name="maxEdits">2</int>
                <!-- the minimum shared prefix when enumerating terms -->
                <int name="minPrefix">1</int>
                <!-- maximum number of inspections per result. -->
                <int name="maxInspections">5</int>
                <!-- minimum length of a query term to be considered for correction -->
                <int name="minQueryLength">4</int>
                <!-- maximum threshold of documents a query term can appear to be considered for correction -->
                <float name="maxQueryFrequency">0.01</float>
                <!-- uncomment this to require suggestions to occur in 1% of the documents
                  <float name="thresholdTokenFrequency">.01</float>
                -->
              </lst>
          
              <!-- a spellchecker that can break or combine words.  See "/spell" handler below for usage -->
              <!--
              <lst name="spellchecker">
                <str name="name">wordbreak</str>
                <str name="classname">solr.WordBreakSolrSpellChecker</str>
                <str name="field">name</str>
                <str name="combineWords">true</str>
                <str name="breakWords">true</str>
                <int name="maxChanges">10</int>
              </lst>
              -->
            </searchComponent>
          
            <!-- A request handler for demonstrating the spellcheck component.  
          
                 NOTE: This is purely as an example.  The whole purpose of the
                 SpellCheckComponent is to hook it into the request handler that
                 handles your normal user queries so that a separate request is
                 not needed to get suggestions.
          
                 IN OTHER WORDS, THERE IS REALLY GOOD CHANCE THE SETUP BELOW IS
                 NOT WHAT YOU WANT FOR YOUR PRODUCTION SYSTEM!
                 
                 See http://wiki.apache.org/solr/SpellCheckComponent for details
                 on the request parameters.
              -->
            <requestHandler name="/spell" class="solr.SearchHandler" startup="lazy">
              <lst name="defaults">
                <!-- Solr will use suggestions from both the 'default' spellchecker
                     and from the 'wordbreak' spellchecker and combine them.
                     collations (re-written queries) can include a combination of
                     corrections from both spellcheckers -->
                <str name="spellcheck.dictionary">default</str>
                <str name="spellcheck">on</str>
                <str name="spellcheck.extendedResults">true</str>
                <str name="spellcheck.count">10</str>
                <str name="spellcheck.alternativeTermCount">5</str>
                <str name="spellcheck.maxResultsForSuggest">5</str>
                <str name="spellcheck.collate">true</str>
                <str name="spellcheck.collateExtendedResults">true</str>
                <str name="spellcheck.maxCollationTries">10</str>
                <str name="spellcheck.maxCollations">5</str>
              </lst>
              <arr name="last-components">
                <str>spellcheck</str>
              </arr>
            </requestHandler>
          
            <!-- Term Vector Component
          
                 http://wiki.apache.org/solr/TermVectorComponent
              -->
            <searchComponent name="tvComponent" class="solr.TermVectorComponent"/>
          
            <!-- A request handler for demonstrating the term vector component
          
                 This is purely as an example.
          
                 In reality you will likely want to add the component to your 
                 already specified request handlers. 
              -->
            <requestHandler name="/tvrh" class="solr.SearchHandler" startup="lazy">
              <lst name="defaults">
                <bool name="tv">true</bool>
              </lst>
              <arr name="last-components">
                <str>tvComponent</str>
              </arr>
            </requestHandler>
          
            <!-- Clustering Component. (Omitted here. See the default Solr example for a typical configuration.) -->
          
            <!-- Terms Component
          
                 http://wiki.apache.org/solr/TermsComponent
          
                 A component to return terms and document frequency of those
                 terms
              -->
            <searchComponent name="terms" class="solr.TermsComponent"/>
          
            <!-- A request handler for demonstrating the terms component -->
            <requestHandler name="/terms" class="solr.SearchHandler" startup="lazy">
              <lst name="defaults">
                <bool name="terms">true</bool>
                <bool name="distrib">false</bool>
              </lst>
              <arr name="components">
                <str>terms</str>
              </arr>
            </requestHandler>
          
          
            <!-- Query Elevation Component
          
                 http://wiki.apache.org/solr/QueryElevationComponent
          
                 a search component that enables you to configure the top
                 results for a given query regardless of the normal lucene
                 scoring.
              -->
            <searchComponent name="elevator" class="solr.QueryElevationComponent" >
              <!-- pick a fieldType to analyze queries -->
              <str name="queryFieldType">string</str>
              <str name="config-file">elevate.xml</str>
            </searchComponent>
          
            <!-- A request handler for demonstrating the elevator component -->
            <requestHandler name="/elevate" class="solr.SearchHandler" startup="lazy">
              <lst name="defaults">
                <str name="echoParams">explicit</str>
              </lst>
              <arr name="last-components">
                <str>elevator</str>
              </arr>
            </requestHandler>
          
            <!-- Highlighting Component
          
                 http://wiki.apache.org/solr/HighlightingParameters
              -->
            <searchComponent class="solr.HighlightComponent" name="highlight">
              <highlighting>
                <!-- Configure the standard fragmenter -->
                <!-- This could most likely be commented out in the "default" case -->
                <fragmenter name="gap"
                            default="true"
                            class="solr.highlight.GapFragmenter">
                  <lst name="defaults">
                    <int name="hl.fragsize">100</int>
                  </lst>
                </fragmenter>
          
                <!-- A regular-expression-based fragmenter 
                     (for sentence extraction) 
                  -->
                <fragmenter name="regex"
                            class="solr.highlight.RegexFragmenter">
                  <lst name="defaults">
                    <!-- slightly smaller fragsizes work better because of slop -->
                    <int name="hl.fragsize">70</int>
                    <!-- allow 50% slop on fragment sizes -->
                    <float name="hl.regex.slop">0.5</float>
                    <!-- a basic sentence pattern -->
                    <str name="hl.regex.pattern">[-\w ,/\n\&quot;&apos;]{20,200}</str>
                  </lst>
                </fragmenter>
          
                <!-- Configure the standard formatter -->
                <formatter name="html"
                           default="true"
                           class="solr.highlight.HtmlFormatter">
                  <lst name="defaults">
                    <str name="hl.simple.pre"><![CDATA[<em>]]></str>
                    <str name="hl.simple.post"><![CDATA[</em>]]></str>
                  </lst>
                </formatter>
          
                <!-- Configure the standard encoder -->
                <encoder name="html"
                         class="solr.highlight.HtmlEncoder" />
          
                <!-- Configure the standard fragListBuilder -->
                <fragListBuilder name="simple"
                                 class="solr.highlight.SimpleFragListBuilder"/>
          
                <!-- Configure the single fragListBuilder -->
                <fragListBuilder name="single"
                                 class="solr.highlight.SingleFragListBuilder"/>
          
                <!-- Configure the weighted fragListBuilder -->
                <fragListBuilder name="weighted"
                                 default="true"
                                 class="solr.highlight.WeightedFragListBuilder"/>
          
                <!-- default tag FragmentsBuilder -->
                <fragmentsBuilder name="default"
                                  default="true"
                                  class="solr.highlight.ScoreOrderFragmentsBuilder">
                  <!-- 
                  <lst name="defaults">
                    <str name="hl.multiValuedSeparatorChar">/</str>
                  </lst>
                  -->
                </fragmentsBuilder>
          
                <!-- multi-colored tag FragmentsBuilder -->
                <fragmentsBuilder name="colored"
                                  class="solr.highlight.ScoreOrderFragmentsBuilder">
                  <lst name="defaults">
                    <str name="hl.tag.pre"><![CDATA[
                         <b style="background:yellow">,<b style="background:lawgreen">,
                         <b style="background:aquamarine">,<b style="background:magenta">,
                         <b style="background:palegreen">,<b style="background:coral">,
                         <b style="background:wheat">,<b style="background:khaki">,
                         <b style="background:lime">,<b style="background:deepskyblue">]]></str>
                    <str name="hl.tag.post"><![CDATA[</b>]]></str>
                  </lst>
                </fragmentsBuilder>
          
                <boundaryScanner name="default"
                                 default="true"
                                 class="solr.highlight.SimpleBoundaryScanner">
                  <lst name="defaults">
                    <str name="hl.bs.maxScan">10</str>
                    <str name="hl.bs.chars">.,!? &#9;&#10;&#13;</str>
                  </lst>
                </boundaryScanner>
          
                <boundaryScanner name="breakIterator"
                                 class="solr.highlight.BreakIteratorBoundaryScanner">
                  <lst name="defaults">
                    <!-- type should be one of CHARACTER, WORD(default), LINE and SENTENCE -->
                    <str name="hl.bs.type">WORD</str>
                    <!-- language and country are used when constructing Locale object.  -->
                    <!-- And the Locale object will be used when getting instance of BreakIterator -->
                    <str name="hl.bs.language">en</str>
                    <str name="hl.bs.country">US</str>
                  </lst>
                </boundaryScanner>
              </highlighting>
            </searchComponent>
          
            <!-- Update Processors
          
                 Chains of Update Processor Factories for dealing with Update
                 Requests can be declared, and then used by name in Update
                 Request Processors
          
                 http://wiki.apache.org/solr/UpdateRequestProcessor
          
              -->
            
            <!-- Add unknown fields to the schema 
            
                 An example field type guessing update processor that will
                 attempt to parse string-typed field values as Booleans, Longs,
                 Doubles, or Dates, and then add schema fields with the guessed
                 field types.  
                 
                 This requires that the schema is both managed and mutable, by
                 declaring schemaFactory as ManagedIndexSchemaFactory, with
                 mutable specified as true. 
                 
                 See http://wiki.apache.org/solr/GuessingFieldTypes
              -->
            <updateRequestProcessorChain name="add-unknown-fields-to-the-schema">
              <!-- UUIDUpdateProcessorFactory will generate an id if none is present in the incoming document -->
              <processor class="solr.UUIDUpdateProcessorFactory" />
          
              <processor class="solr.LogUpdateProcessorFactory"/>
              <processor class="solr.DistributedUpdateProcessorFactory"/>
              <processor class="solr.RemoveBlankFieldUpdateProcessorFactory"/>
              <processor class="solr.FieldNameMutatingUpdateProcessorFactory">
                <str name="pattern">[^\w-\.]</str>
                <str name="replacement">_</str>
              </processor>
              <processor class="solr.ParseBooleanFieldUpdateProcessorFactory"/>
              <processor class="solr.ParseLongFieldUpdateProcessorFactory"/>
              <processor class="solr.ParseDoubleFieldUpdateProcessorFactory"/>
              <processor class="solr.ParseDateFieldUpdateProcessorFactory">
                <arr name="format">
                  <str>yyyy-MM-dd'T'HH:mm:ss.SSSZ</str>
                  <str>yyyy-MM-dd'T'HH:mm:ss,SSSZ</str>
                  <str>yyyy-MM-dd'T'HH:mm:ss.SSS</str>
                  <str>yyyy-MM-dd'T'HH:mm:ss,SSS</str>
                  <str>yyyy-MM-dd'T'HH:mm:ssZ</str>
                  <str>yyyy-MM-dd'T'HH:mm:ss</str>
                  <str>yyyy-MM-dd'T'HH:mmZ</str>
                  <str>yyyy-MM-dd'T'HH:mm</str>
                  <str>yyyy-MM-dd HH:mm:ss.SSSZ</str>
                  <str>yyyy-MM-dd HH:mm:ss,SSSZ</str>
                  <str>yyyy-MM-dd HH:mm:ss.SSS</str>
                  <str>yyyy-MM-dd HH:mm:ss,SSS</str>
                  <str>yyyy-MM-dd HH:mm:ssZ</str>
                  <str>yyyy-MM-dd HH:mm:ss</str>
                  <str>yyyy-MM-dd HH:mmZ</str>
                  <str>yyyy-MM-dd HH:mm</str>
                  <str>yyyy-MM-dd</str>
                </arr>
              </processor>
              <processor class="solr.AddSchemaFieldsUpdateProcessorFactory">
                <str name="defaultFieldType">strings</str>
                <lst name="typeMapping">
                  <str name="valueClass">java.lang.Boolean</str>
                  <str name="fieldType">booleans</str>
                </lst>
                <lst name="typeMapping">
                  <str name="valueClass">java.util.Date</str>
                  <str name="fieldType">tdates</str>
                </lst>
                <lst name="typeMapping">
                  <str name="valueClass">java.lang.Long</str>
                  <str name="valueClass">java.lang.Integer</str>
                  <str name="fieldType">tlongs</str>
                </lst>
                <lst name="typeMapping">
                  <str name="valueClass">java.lang.Number</str>
                  <str name="fieldType">tdoubles</str>
                </lst>
              </processor>
              <processor class="solr.RunUpdateProcessorFactory"/>
            </updateRequestProcessorChain>
          
            <updateRequestProcessorChain name="classification">
              <processor class="solr.ClassificationUpdateProcessorFactory">
                <str name="inputFields">title_t^1.5,author_s</str>
                <str name="classField">cat_s</str>
                <str name="algorithm">bayes</str>
              </processor>
            </updateRequestProcessorChain>
          
            <!-- Deduplication
          
                 An example dedup update processor that creates the "id" field
                 on the fly based on the hash code of some other fields.  This
                 example has overwriteDupes set to false since we are using the
                 id field as the signatureField and Solr will maintain
                 uniqueness based on that anyway.  
                 
              -->
            <!--
               <updateRequestProcessorChain name="dedupe">
                 <processor class="solr.processor.SignatureUpdateProcessorFactory">
                   <bool name="enabled">true</bool>
                   <str name="signatureField">id</str>
                   <bool name="overwriteDupes">false</bool>
                   <str name="fields">name,features,cat</str>
                   <str name="signatureClass">solr.processor.Lookup3Signature</str>
                 </processor>
                 <processor class="solr.LogUpdateProcessorFactory" />
                 <processor class="solr.RunUpdateProcessorFactory" />
               </updateRequestProcessorChain>
              -->
          
            <!-- Language identification
          
                 This example update chain identifies the language of the incoming
                 documents using the langid contrib. The detected language is
                 written to field language_s. No field name mapping is done.
                 The fields used for detection are text, title, subject and description,
                 making this example suitable for detecting languages form full-text
                 rich documents injected via ExtractingRequestHandler.
                 See more about langId at http://wiki.apache.org/solr/LanguageDetection
              -->
            <!--
             <updateRequestProcessorChain name="langid">
               <processor class="org.apache.solr.update.processor.TikaLanguageIdentifierUpdateProcessorFactory">
                 <str name="langid.fl">text,title,subject,description</str>
                 <str name="langid.langField">language_s</str>
                 <str name="langid.fallback">en</str>
               </processor>
               <processor class="solr.LogUpdateProcessorFactory" />
               <processor class="solr.RunUpdateProcessorFactory" />
             </updateRequestProcessorChain>
            -->
          
            <!-- Script update processor
          
              This example hooks in an update processor implemented using JavaScript.
          
              See more about the script update processor at http://wiki.apache.org/solr/ScriptUpdateProcessor
            -->
            <!--
              <updateRequestProcessorChain name="script">
                <processor class="solr.StatelessScriptUpdateProcessorFactory">
                  <str name="script">update-script.js</str>
                  <lst name="params">
                    <str name="config_param">example config parameter</str>
                  </lst>
                </processor>
                <processor class="solr.RunUpdateProcessorFactory" />
              </updateRequestProcessorChain>
            -->
          
            <!-- Response Writers
          
                 http://wiki.apache.org/solr/QueryResponseWriter
          
                 Request responses will be written using the writer specified by
                 the 'wt' request parameter matching the name of a registered
                 writer.
          
                 The "default" writer is the default and will be used if 'wt' is
                 not specified in the request.
              -->
            <!-- The following response writers are implicitly configured unless
                 overridden...
              -->
            <!--
               <queryResponseWriter name="xml" 
                                    default="true"
                                    class="solr.XMLResponseWriter" />
               <queryResponseWriter name="json" class="solr.JSONResponseWriter"/>
               <queryResponseWriter name="python" class="solr.PythonResponseWriter"/>
               <queryResponseWriter name="ruby" class="solr.RubyResponseWriter"/>
               <queryResponseWriter name="php" class="solr.PHPResponseWriter"/>
               <queryResponseWriter name="phps" class="solr.PHPSerializedResponseWriter"/>
               <queryResponseWriter name="csv" class="solr.CSVResponseWriter"/>
               <queryResponseWriter name="schema.xml" class="solr.SchemaXmlResponseWriter"/>
              -->
          
            <queryResponseWriter name="json" class="solr.JSONResponseWriter">
              <!-- For the purposes of the tutorial, JSON responses are written as
               plain text so that they are easy to read in *any* browser.
               If you expect a MIME type of "application/json" just remove this override.
              -->
              <str name="content-type">text/plain; charset=UTF-8</str>
            </queryResponseWriter>
          
            <!--
               Custom response writers can be declared as needed...
              -->
            <queryResponseWriter name="velocity" class="solr.VelocityResponseWriter" startup="lazy">
              <str name="template.base.dir">${velocity.template.base.dir:}</str>
              <str name="solr.resource.loader.enabled">${velocity.solr.resource.loader.enabled:true}</str>
              <str name="params.resource.loader.enabled">${velocity.params.resource.loader.enabled:false}</str>
            </queryResponseWriter>
          
            <!-- XSLT response writer transforms the XML output by any xslt file found
                 in Solr's conf/xslt directory.  Changes to xslt files are checked for
                 every xsltCacheLifetimeSeconds.  
              -->
            <queryResponseWriter name="xslt" class="solr.XSLTResponseWriter">
              <int name="xsltCacheLifetimeSeconds">5</int>
            </queryResponseWriter>
          
            <!-- Query Parsers
          
                 https://cwiki.apache.org/confluence/display/solr/Query+Syntax+and+Parsing
          
                 Multiple QParserPlugins can be registered by name, and then
                 used in either the "defType" param for the QueryComponent (used
                 by SearchHandler) or in LocalParams
              -->
            <!-- example of registering a query parser -->
            <!--
               <queryParser name="myparser" class="com.mycompany.MyQParserPlugin"/>
              -->
          
            <!-- Function Parsers
          
                 http://wiki.apache.org/solr/FunctionQuery
          
                 Multiple ValueSourceParsers can be registered by name, and then
                 used as function names when using the "func" QParser.
              -->
            <!-- example of registering a custom function parser  -->
            <!--
               <valueSourceParser name="myfunc" 
                                  class="com.mycompany.MyValueSourceParser" />
              -->
          
          
            <!-- Document Transformers
                 http://wiki.apache.org/solr/DocTransformers
              -->
            <!--
               Could be something like:
               <transformer name="db" class="com.mycompany.LoadFromDatabaseTransformer" >
                 <int name="connection">jdbc://....</int>
               </transformer>
               
               To add a constant value to all docs, use:
               <transformer name="mytrans2" class="org.apache.solr.response.transform.ValueAugmenterFactory" >
                 <int name="value">5</int>
               </transformer>
               
               If you want the user to still be able to change it with _value:something_ use this:
               <transformer name="mytrans3" class="org.apache.solr.response.transform.ValueAugmenterFactory" >
                 <double name="defaultValue">5</double>
               </transformer>
          
                If you are using the QueryElevationComponent, you may wish to mark documents that get boosted.  The
                EditorialMarkerFactory will do exactly that:
               <transformer name="qecBooster" class="org.apache.solr.response.transform.EditorialMarkerFactory" />
              -->
          </config>
          
          Show
          tomas0413 Tomas Ramanauskas added a comment - Not sure, I can attach the file here. This is the configuration file: ./server/solr/demo/conf/solrconfig.xml <?xml version= "1.0" encoding= "UTF-8" ?> <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the "License" ); you may not use this file except in compliance with the License. You may obtain a copy of the License at http: //www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> <!-- For more details about configurations options that may appear in this file, see http: //wiki.apache.org/solr/SolrConfigXml. --> <config> <!-- In all configuration below, a prefix of "solr." for class names is an alias that causes solr to search appropriate packages, including org.apache.solr.(search|update|request|core|analysis) You may also specify a fully qualified Java classname if you have your own custom plugins. --> <!-- Controls what version of Lucene various components of Solr adhere to. Generally, you want to use the latest version to get all bug fixes and improvements. It is highly recommended that you fully re-index after changing this setting as it can affect both how text is indexed and queried. --> <luceneMatchVersion>6.1.0</luceneMatchVersion> <!-- <lib/> directives can be used to instruct Solr to load any Jars identified and use them to resolve any "plugins" specified in your solrconfig.xml or schema.xml (ie: Analyzers, Request Handlers, etc...). All directories and paths are resolved relative to the instanceDir. Please note that <lib/> directives are processed in the order that they appear in your solrconfig.xml file, and are "stacked" on top of each other when building a ClassLoader - so if you have plugin jars with dependencies on other jars, the "lower level" dependency jars should be loaded first. If a "./lib" directory exists in your instanceDir, all files found in it are included as if you had used the following syntax... <lib dir= "./lib" /> --> <!-- A 'dir' option by itself adds any files found in the directory to the classpath, this is useful for including all jars in a directory. When a 'regex' is specified in addition to a 'dir', only the files in that directory which completely match the regex (anchored on both ends) will be included. If a 'dir' option (with or without a regex) is used and nothing is found that matches, a warning will be logged. The examples below can be used to load some solr-contribs along with their external dependencies. --> <lib dir= "${solr.install.dir:../../../..}/contrib/extraction/lib" regex= ".*\.jar" /> <lib dir= "${solr.install.dir:../../../..}/dist/" regex= "solr-cell-\d.*\.jar" /> <lib dir= "${solr.install.dir:../../../..}/contrib/clustering/lib/" regex= ".*\.jar" /> <lib dir= "${solr.install.dir:../../../..}/dist/" regex= "solr-clustering-\d.*\.jar" /> <lib dir= "${solr.install.dir:../../../..}/contrib/langid/lib/" regex= ".*\.jar" /> <lib dir= "${solr.install.dir:../../../..}/dist/" regex= "solr-langid-\d.*\.jar" /> <lib dir= "${solr.install.dir:../../../..}/contrib/velocity/lib" regex= ".*\.jar" /> <lib dir= "${solr.install.dir:../../../..}/dist/" regex= "solr-velocity-\d.*\.jar" /> <!-- an exact 'path' can be used instead of a 'dir' to specify a specific jar file. This will cause a serious error to be logged if it can't be loaded. --> <!-- <lib path= "../a-jar-that-does-not-exist.jar" /> --> <!-- Data Directory Used to specify an alternate directory to hold all index data other than the default ./data under the Solr home. If replication is in use, this should match the replication configuration. --> <dataDir>${solr.data.dir:}</dataDir> <!-- The DirectoryFactory to use for indexes. solr.StandardDirectoryFactory is filesystem based and tries to pick the best implementation for the current JVM and platform. solr.NRTCachingDirectoryFactory, the default , wraps solr.StandardDirectoryFactory and caches small files in memory for better NRT performance. One can force a particular implementation via solr.MMapDirectoryFactory, solr.NIOFSDirectoryFactory, or solr.SimpleFSDirectoryFactory. solr.RAMDirectoryFactory is memory based, not persistent, and doesn't work with replication. --> <directoryFactory name= "DirectoryFactory" class= "${solr.directoryFactory:solr.NRTCachingDirectoryFactory}" /> <!-- The CodecFactory for defining the format of the inverted index. The default implementation is SchemaCodecFactory, which is the official Lucene index format, but hooks into the schema to provide per-field customization of the postings lists and per-document values in the fieldType element (postingsFormat/docValuesFormat). Note that most of the alternative implementations are experimental, so if you choose to customize the index format, it's a good idea to convert back to the official format e.g. via IndexWriter.addIndexes(IndexReader) before upgrading to a newer version to avoid unnecessary reindexing. A "compressionMode" string element can be added to <codecFactory> to choose between the existing compression modes in the default codec: "BEST_SPEED" ( default ) or "BEST_COMPRESSION" . --> <codecFactory class= "solr.SchemaCodecFactory" /> <!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Index Config - These settings control low-level behavior of indexing Most example settings here show the default value, but are commented out, to more easily see where customizations have been made. Note: This replaces <indexDefaults> and <mainIndex> from older versions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ --> <indexConfig> <!-- maxFieldLength was removed in 4.0. To get similar behavior, include a LimitTokenCountFilterFactory in your fieldType definition. E.g. <filter class= "solr.LimitTokenCountFilterFactory" maxTokenCount= "10000" /> --> <!-- Maximum time to wait for a write lock (ms) for an IndexWriter. Default: 1000 --> <!-- <writeLockTimeout>1000</writeLockTimeout> --> <!-- Expert: Enabling compound file will use less files for the index, using fewer file descriptors on the expense of performance decrease. Default in Lucene is " true " . Default in Solr is " false " (since 3.6) --> <!-- <useCompoundFile> false </useCompoundFile> --> <!-- ramBufferSizeMB sets the amount of RAM that may be used by Lucene indexing for buffering added documents and deletions before they are flushed to the Directory. maxBufferedDocs sets a limit on the number of documents buffered before flushing. If both ramBufferSizeMB and maxBufferedDocs is set, then Lucene will flush based on whichever limit is hit first. --> <!-- <ramBufferSizeMB>100</ramBufferSizeMB> --> <!-- <maxBufferedDocs>1000</maxBufferedDocs> --> <!-- Expert: Merge Policy The Merge Policy in Lucene controls how merging of segments is done. The default since Solr/Lucene 3.3 is TieredMergePolicy. The default since Lucene 2.3 was the LogByteSizeMergePolicy, Even older versions of Lucene used LogDocMergePolicy. --> <!-- <mergePolicyFactory class= "org.apache.solr.index.TieredMergePolicyFactory" > < int name= "maxMergeAtOnce" >10</ int > < int name= "segmentsPerTier" >10</ int > < double name= "noCFSRatio" >0.1</ double > </mergePolicyFactory> --> <!-- Expert: Merge Scheduler The Merge Scheduler in Lucene controls how merges are performed. The ConcurrentMergeScheduler (Lucene 2.3 default ) can perform merges in the background using separate threads. The SerialMergeScheduler (Lucene 2.2 default ) does not. --> <!-- <mergeScheduler class= "org.apache.lucene.index.ConcurrentMergeScheduler" /> --> <!-- LockFactory This option specifies which Lucene LockFactory implementation to use. single = SingleInstanceLockFactory - suggested for a read-only index or when there is no possibility of another process trying to modify the index. native = NativeFSLockFactory - uses OS native file locking. Do not use when multiple solr webapps in the same JVM are attempting to share a single index. simple = SimpleFSLockFactory - uses a plain file for locking Defaults: ' native ' is default for Solr3.6 and later, otherwise 'simple' is the default More details on the nuances of each LockFactory... http: //wiki.apache.org/lucene-java/AvailableLockFactories --> <lockType>${solr.lock.type: native }</lockType> <!-- Commit Deletion Policy Custom deletion policies can be specified here. The class must implement org.apache.lucene.index.IndexDeletionPolicy. The default Solr IndexDeletionPolicy implementation supports deleting index commit points on number of commits, age of commit point and optimized status. The latest commit point should always be preserved regardless of the criteria. --> <!-- <deletionPolicy class= "solr.SolrDeletionPolicy" > --> <!-- The number of commit points to be kept --> <!-- <str name= "maxCommitsToKeep" >1</str> --> <!-- The number of optimized commit points to be kept --> <!-- <str name= "maxOptimizedCommitsToKeep" >0</str> --> <!-- Delete all commit points once they have reached the given age. Supports DateMathParser syntax e.g. --> <!-- <str name= "maxCommitAge" >30MINUTES</str> <str name= "maxCommitAge" >1DAY</str> --> <!-- </deletionPolicy> --> <!-- Lucene Infostream To aid in advanced debugging, Lucene provides an "InfoStream" of detailed information when indexing. Setting The value to true will instruct the underlying Lucene IndexWriter to write its debugging info the specified file --> <!-- <infoStream file= "INFOSTREAM.txt" > false </infoStream> --> </indexConfig> <!-- JMX This example enables JMX if and only if an existing MBeanServer is found, use this if you want to configure JMX through JVM parameters. Remove this to disable exposing Solr configuration and statistics to JMX. For more details see http: //wiki.apache.org/solr/SolrJmx --> <jmx /> <!-- If you want to connect to a particular server, specify the agentId --> <!-- <jmx agentId= "myAgent" /> --> <!-- If you want to start a new MBeanServer, specify the serviceUrl --> <!-- <jmx serviceUrl= "service:jmx:rmi: ///jndi/rmi://localhost:9999/solr" /> --> <!-- The default high-performance update handler --> <updateHandler class= "solr.DirectUpdateHandler2" > <!-- Enables a transaction log, used for real-time get, durability, and and solr cloud replica recovery. The log can grow as big as uncommitted changes to the index, so use of a hard autoCommit is recommended (see below). "dir" - the target directory for transaction logs, defaults to the solr data directory. "numVersionBuckets" - sets the number of buckets used to keep track of max version values when checking for re-ordered updates; increase this value to reduce the cost of synchronizing access to version buckets during high-volume indexing, this requires 8 bytes ( long ) * numVersionBuckets of heap space per Solr core. --> <updateLog> <str name= "dir" >${solr.ulog.dir:}</str> < int name= "numVersionBuckets" >${solr.ulog.numVersionBuckets:65536}</ int > </updateLog> <!-- AutoCommit Perform a hard commit automatically under certain conditions. Instead of enabling autoCommit, consider using "commitWithin" when adding documents. http: //wiki.apache.org/solr/UpdateXmlMessages maxDocs - Maximum number of documents to add since the last commit before automatically triggering a new commit. maxTime - Maximum amount of time in ms that is allowed to pass since a document was added before automatically triggering a new commit. openSearcher - if false , the commit causes recent index changes to be flushed to stable storage, but does not cause a new searcher to be opened to make those changes visible. If the updateLog is enabled, then it's highly recommended to have some sort of hard autoCommit to limit the log size. --> <autoCommit> <maxTime>${solr.autoCommit.maxTime:15000}</maxTime> <openSearcher> false </openSearcher> </autoCommit> <!-- softAutoCommit is like autoCommit except it causes a 'soft' commit which only ensures that changes are visible but does not ensure that data is synced to disk. This is faster and more near-realtime friendly than a hard commit. --> <autoSoftCommit> <maxTime>${solr.autoSoftCommit.maxTime:-1}</maxTime> </autoSoftCommit> <!-- Update Related Event Listeners Various IndexWriter related events can trigger Listeners to take actions. postCommit - fired after every commit or optimize command postOptimize - fired after every optimize command --> <!-- The RunExecutableListener executes an external command from a hook such as postCommit or postOptimize. exe - the name of the executable to run dir - dir to use as the current working directory. ( default = "." ) wait - the calling thread waits until the executable returns. ( default = " true " ) args - the arguments to pass to the program. ( default is none) env - environment variables to set. ( default is none) --> <!-- This example shows how RunExecutableListener could be used with the script based replication... http: //wiki.apache.org/solr/CollectionDistribution --> <!-- <listener event= "postCommit" class= "solr.RunExecutableListener" > <str name= "exe" >solr/bin/snapshooter</str> <str name= "dir" >.</str> <bool name= "wait" > true </bool> <arr name= "args" > <str>arg1</str> <str>arg2</str> </arr> <arr name= "env" > <str>MYVAR=val1</str> </arr> </listener> --> </updateHandler> <!-- IndexReaderFactory Use the following format to specify a custom IndexReaderFactory, which allows for alternate IndexReader implementations. ** Experimental Feature ** Please note - Using a custom IndexReaderFactory may prevent certain other features from working. The API to IndexReaderFactory may change without warning or may even be removed from future releases if the problems cannot be resolved. ** Features that may not work with custom IndexReaderFactory ** The ReplicationHandler assumes a disk-resident index. Using a custom IndexReader implementation may cause incompatibility with ReplicationHandler and may cause replication to not work correctly. See SOLR-1366 for details. --> <!-- <indexReaderFactory name= "IndexReaderFactory" class= " package .class" > <str name= "someArg" >Some Value</str> </indexReaderFactory > --> <!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Query section - these settings control query time things like caches ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ --> <query> <!-- Max Boolean Clauses Maximum number of clauses in each BooleanQuery, an exception is thrown if exceeded. ** WARNING ** This option actually modifies a global Lucene property that will affect all SolrCores. If multiple solrconfig.xml files disagree on this property, the value at any given moment will be based on the last SolrCore to be initialized. --> <maxBooleanClauses>1024</maxBooleanClauses> <!-- Solr Internal Query Caches There are two implementations of cache available for Solr, LRUCache, based on a synchronized LinkedHashMap, and FastLRUCache, based on a ConcurrentHashMap. FastLRUCache has faster gets and slower puts in single threaded operation and thus is generally faster than LRUCache when the hit ratio of the cache is high (> 75%), and may be faster under other scenarios on multi-cpu systems. --> <!-- Filter Cache Cache used by SolrIndexSearcher for filters (DocSets), unordered sets of *all* documents that match a query. When a new searcher is opened, its caches may be prepopulated or "autowarmed" using data from caches in the old searcher. autowarmCount is the number of items to prepopulate. For LRUCache, the autowarmed items will be the most recently accessed items. Parameters: class - the SolrCache implementation LRUCache or (LRUCache or FastLRUCache) size - the maximum number of entries in the cache initialSize - the initial capacity (number of entries) of the cache. (see java.util.HashMap) autowarmCount - the number of entries to prepopulate from and old cache. --> <filterCache class= "solr.FastLRUCache" size= "512" initialSize= "512" autowarmCount= "0" /> <!-- Query Result Cache Caches results of searches - ordered lists of document ids (DocList) based on a query, a sort, and the range of documents requested. Additional supported parameter by LRUCache: maxRamMB - the maximum amount of RAM (in MB) that this cache is allowed to occupy --> <queryResultCache class= "solr.LRUCache" size= "512" initialSize= "512" autowarmCount= "0" /> <!-- Document Cache Caches Lucene Document objects (the stored fields for each document). Since Lucene internal document ids are transient , this cache will not be autowarmed. --> <documentCache class= "solr.LRUCache" size= "512" initialSize= "512" autowarmCount= "0" /> <!-- custom cache currently used by block join --> <cache name= "perSegFilter" class= "solr.search.LRUCache" size= "10" initialSize= "0" autowarmCount= "10" regenerator= "solr.NoOpRegenerator" /> <!-- Field Value Cache Cache used to hold field values that are quickly accessible by document id. The fieldValueCache is created by default even if not configured here. --> <!-- <fieldValueCache class= "solr.FastLRUCache" size= "512" autowarmCount= "128" showItems= "32" /> --> <!-- Custom Cache Example of a generic cache. These caches may be accessed by name through SolrIndexSearcher.getCache(),cacheLookup(), and cacheInsert(). The purpose is to enable easy caching of user/application level data. The regenerator argument should be specified as an implementation of solr.CacheRegenerator if autowarming is desired. --> <!-- <cache name= "myUserCache" class= "solr.LRUCache" size= "4096" initialSize= "1024" autowarmCount= "1024" regenerator= "com.mycompany.MyRegenerator" /> --> <!-- Lazy Field Loading If true , stored fields that are not requested will be loaded lazily. This can result in a significant speed improvement if the usual case is to not load all stored fields, especially if the skipped fields are large compressed text fields. --> <enableLazyFieldLoading> true </enableLazyFieldLoading> <!-- Use Filter For Sorted Query A possible optimization that attempts to use a filter to satisfy a search. If the requested sort does not include score, then the filterCache will be checked for a filter matching the query. If found, the filter will be used as the source of document ids, and then the sort will be applied to that. For most situations, this will not be useful unless you frequently get the same search repeatedly with different sort options, and none of them ever use "score" --> <!-- <useFilterForSortedQuery> true </useFilterForSortedQuery> --> <!-- Result Window Size An optimization for use with the queryResultCache. When a search is requested, a superset of the requested number of document ids are collected. For example, if a search for a particular query requests matching documents 10 through 19, and queryWindowSize is 50, then documents 0 through 49 will be collected and cached. Any further requests in that range can be satisfied via the cache. --> <queryResultWindowSize>20</queryResultWindowSize> <!-- Maximum number of documents to cache for any entry in the queryResultCache. --> <queryResultMaxDocsCached>200</queryResultMaxDocsCached> <!-- Query Related Event Listeners Various IndexSearcher related events can trigger Listeners to take actions. newSearcher - fired whenever a new searcher is being prepared and there is a current searcher handling requests (aka registered). It can be used to prime certain caches to prevent long request times for certain requests. firstSearcher - fired whenever a new searcher is being prepared but there is no current registered searcher to handle requests or to gain autowarming data from. --> <!-- QuerySenderListener takes an array of NamedList and executes a local query request for each NamedList in sequence. --> <listener event= "newSearcher" class= "solr.QuerySenderListener" > <arr name= "queries" > <!-- <lst><str name= "q" >solr</str><str name= "sort" >price asc</str></lst> <lst><str name= "q" >rocks</str><str name= "sort" >weight asc</str></lst> --> </arr> </listener> <listener event= "firstSearcher" class= "solr.QuerySenderListener" > <arr name= "queries" > <!-- <lst> <str name= "q" > static firstSearcher warming in solrconfig.xml</str> </lst> --> </arr> </listener> <!-- Use Cold Searcher If a search request comes in and there is no current registered searcher, then immediately register the still warming searcher and use it. If " false " then all requests will block until the first searcher is done warming. --> <useColdSearcher> false </useColdSearcher> <!-- Max Warming Searchers Maximum number of searchers that may be warming in the background concurrently. An error is returned if this limit is exceeded. Recommend values of 1-2 for read-only slaves, higher for masters w/o cache warming. --> <maxWarmingSearchers>2</maxWarmingSearchers> </query> <!-- Request Dispatcher This section contains instructions for how the SolrDispatchFilter should behave when processing requests for this SolrCore. handleSelect is a legacy option that affects the behavior of requests such as /select?qt=XXX handleSelect= " true " will cause the SolrDispatchFilter to process the request and dispatch the query to a handler specified by the "qt" param, assuming "/select" isn't already registered. handleSelect= " false " will cause the SolrDispatchFilter to ignore "/select" requests, resulting in a 404 unless a handler is explicitly registered with the name "/select" handleSelect= " true " is not recommended for new users, but is the default for backwards compatibility --> <requestDispatcher handleSelect= " false " > <!-- Request Parsing These settings indicate how Solr Requests may be parsed, and what restrictions may be placed on the ContentStreams from those requests enableRemoteStreaming - enables use of the stream.file and stream.url parameters for specifying remote streams. multipartUploadLimitInKB - specifies the max size (in KiB) of Multipart File Uploads that Solr will allow in a Request. formdataUploadLimitInKB - specifies the max size (in KiB) of form data (application/x-www-form-urlencoded) sent via POST. You can use POST to pass request parameters not fitting into the URL. addHttpRequestToContext - if set to true , it will instruct the requestParsers to include the original HttpServletRequest object in the context map of the SolrQueryRequest under the key "httpRequest" . It will not be used by any of the existing Solr components, but may be useful when developing custom plugins. *** WARNING *** The settings below authorize Solr to fetch remote files, You should make sure your system has some authentication before using enableRemoteStreaming= " true " --> <requestParsers enableRemoteStreaming= " true " multipartUploadLimitInKB= "2048000" formdataUploadLimitInKB= "2048" addHttpRequestToContext= " false " /> <!-- HTTP Caching Set HTTP caching related parameters ( for proxy caches and clients). The options below instruct Solr not to output any HTTP Caching related headers --> <httpCaching never304= " true " /> <!-- If you include a <cacheControl> directive, it will be used to generate a Cache-Control header (as well as an Expires header if the value contains "max-age=" ) By default , no Cache-Control header is generated. You can use the <cacheControl> option even if you have set never304= " true " --> <!-- <httpCaching never304= " true " > <cacheControl>max-age=30, public </cacheControl> </httpCaching> --> <!-- To enable Solr to respond with automatically generated HTTP Caching headers, and to response to Cache Validation requests correctly, set the value of never304= " false " This will cause Solr to generate Last-Modified and ETag headers based on the properties of the Index. The following options can also be specified to affect the values of these headers... lastModFrom - the default value is "openTime" which means the Last-Modified value (and validation against If-Modified-Since requests) will all be relative to when the current Searcher was opened. You can change it to lastModFrom= "dirLastMod" if you want the value to exactly correspond to when the physical index was last modified. etagSeed= "..." is an option you can change to force the ETag header (and validation against If-None-Match requests) to be different even if the index has not changed (ie: when making significant changes to your config file) (lastModifiedFrom and etagSeed are both ignored if you use the never304= " true " option) --> <!-- <httpCaching lastModifiedFrom= "openTime" etagSeed= "Solr" > <cacheControl>max-age=30, public </cacheControl> </httpCaching> --> </requestDispatcher> <!-- Request Handlers http: //wiki.apache.org/solr/SolrRequestHandler Incoming queries will be dispatched to a specific handler by name based on the path specified in the request. Legacy behavior: If the request path uses "/select" but no Request Handler has that name, and if handleSelect= " true " has been specified in the requestDispatcher, then the Request Handler is dispatched based on the qt parameter. Handlers without a leading '/' are accessed this way like so: http: //host/app/[core/]select?qt=name If no qt is given, then the requestHandler that declares default = " true " will be used or the one named "standard" . If a Request Handler is declared with startup= "lazy" , then it will not be initialized until the first request that uses it. --> <!-- SearchHandler http: //wiki.apache.org/solr/SearchHandler For processing Search Queries, the primary Request Handler provided with Solr is "SearchHandler" It delegates to a sequent of SearchComponents (see below) and supports distributed queries across multiple shards --> <requestHandler name= "/select" class= "solr.SearchHandler" > <!-- default values for query parameters can be specified, these will be overridden by parameters in the request --> <lst name= "defaults" > <str name= "echoParams" >explicit</str> < int name= "rows" >10</ int > <!-- <str name= "df" >text</str> --> </lst> <!-- In addition to defaults, "appends" params can be specified to identify values which should be appended to the list of multi-val params from the query (or the existing "defaults" ). --> <!-- In this example, the param "fq=instock: true " would be appended to any query time fq params the user may specify, as a mechanism for partitioning the index, independent of any user selected filtering that may also be desired (perhaps as a result of faceted searching). NOTE: there is *absolutely* nothing a client can do to prevent these "appends" values from being used, so don't use this mechanism unless you are sure you always want it. --> <!-- <lst name= "appends" > <str name= "fq" >inStock: true </str> </lst> --> <!-- "invariants" are a way of letting the Solr maintainer lock down the options available to Solr clients. Any params values specified here are used regardless of what values may be specified in either the query, the "defaults" , or the "appends" params. In this example, the facet.field and facet.query params would be fixed, limiting the facets clients can use. Faceting is not turned on by default - but if the client does specify facet= true in the request, these are the only facets they will be able to see counts for ; regardless of what other facet.field or facet.query params they may specify. NOTE: there is *absolutely* nothing a client can do to prevent these "invariants" values from being used, so don't use this mechanism unless you are sure you always want it. --> <!-- <lst name= "invariants" > <str name= "facet.field" >cat</str> <str name= "facet.field" >manu_exact</str> <str name= "facet.query" >price:[* TO 500]</str> <str name= "facet.query" >price:[500 TO *]</str> </lst> --> <!-- If the default list of SearchComponents is not desired, that list can either be overridden completely, or components can be prepended or appended to the default list. (see below) --> <!-- <arr name= "components" > <str>nameOfCustomComponent1</str> <str>nameOfCustomComponent2</str> </arr> --> </requestHandler> <!-- A request handler that returns indented JSON by default --> <requestHandler name= "/query" class= "solr.SearchHandler" > <lst name= "defaults" > <str name= "echoParams" >explicit</str> <str name= "wt" >json</str> <str name= "indent" > true </str> </lst> </requestHandler> <!-- A Robust Example This example SearchHandler declaration shows off usage of the SearchHandler with many defaults declared Note that multiple instances of the same Request Handler (SearchHandler) can be registered multiple times with different names (and different init parameters) --> <requestHandler name= "/browse" class= "solr.SearchHandler" useParams= "query,facets,velocity,browse" > <lst name= "defaults" > <str name= "echoParams" >explicit</str> </lst> </requestHandler> <initParams path= "/update/**,/query,/select,/tvrh,/elevate,/spell,/browse" > <lst name= "defaults" > <str name= "df" >_text_</str> </lst> </initParams> <initParams path= "/update/**" > <lst name= "defaults" > <str name= "update.chain" >add-unknown-fields-to-the-schema</str> </lst> </initParams> <!-- Solr Cell Update Request Handler http: //wiki.apache.org/solr/ExtractingRequestHandler --> <requestHandler name= "/update/extract" startup= "lazy" class= "solr.extraction.ExtractingRequestHandler" > <lst name= "defaults" > <str name= "lowernames" > true </str> <str name= "fmap.meta" >ignored_</str> <str name= "fmap.content" >_text_</str> </lst> </requestHandler> <!-- Field Analysis Request Handler RequestHandler that provides much the same functionality as analysis.jsp. Provides the ability to specify multiple field types and field names in the same request and outputs index-time and query-time analysis for each of them. Request parameters are: analysis.fieldname - field name whose analyzers are to be used analysis.fieldtype - field type whose analyzers are to be used analysis.fieldvalue - text for index-time analysis q (or analysis.q) - text for query time analysis analysis.showmatch ( true | false ) - When set to true and when query analysis is performed, the produced tokens of the field value analysis will be marked as "matched" for every token that is produces by the query analysis --> <requestHandler name= "/analysis/field" startup= "lazy" class= "solr.FieldAnalysisRequestHandler" /> <!-- Document Analysis Handler http: //wiki.apache.org/solr/AnalysisRequestHandler An analysis handler that provides a breakdown of the analysis process of provided documents. This handler expects a (single) content stream with the following format: <docs> <doc> <field name= "id" >1</field> <field name= "name" >The Name</field> <field name= "text" >The Text Value</field> </doc> <doc>...</doc> <doc>...</doc> ... </docs> Note: Each document must contain a field which serves as the unique key. This key is used in the returned response to associate an analysis breakdown to the analyzed document. Like the FieldAnalysisRequestHandler, this handler also supports query analysis by sending either an "analysis.query" or "q" request parameter that holds the query text to be analyzed. It also supports the "analysis.showmatch" parameter which when set to true , all field tokens that match the query tokens will be marked as a "match" . --> <requestHandler name= "/analysis/document" class= "solr.DocumentAnalysisRequestHandler" startup= "lazy" /> <!-- Echo the request contents back to the client --> <requestHandler name= "/debug/dump" class= "solr.DumpRequestHandler" > <lst name= "defaults" > <str name= "echoParams" >explicit</str> <str name= "echoHandler" > true </str> </lst> </requestHandler> <!-- Search Components Search components are registered to SolrCore and used by instances of SearchHandler (which can access them by name) By default , the following components are available: <searchComponent name= "query" class= "solr.QueryComponent" /> <searchComponent name= "facet" class= "solr.FacetComponent" /> <searchComponent name= "mlt" class= "solr.MoreLikeThisComponent" /> <searchComponent name= "highlight" class= "solr.HighlightComponent" /> <searchComponent name= "stats" class= "solr.StatsComponent" /> <searchComponent name= "debug" class= "solr.DebugComponent" /> Default configuration in a requestHandler would look like: <arr name= "components" > <str>query</str> <str>facet</str> <str>mlt</str> <str>highlight</str> <str>stats</str> <str>debug</str> </arr> If you register a searchComponent to one of the standard names, that will be used instead of the default . To insert components before or after the 'standard' components, use: <arr name= "first-components" > <str>myFirstComponentName</str> </arr> <arr name= "last-components" > <str>myLastComponentName</str> </arr> NOTE: The component registered with the name "debug" will always be executed after the "last-components" --> <!-- Spell Check The spell check component can return a list of alternative spelling suggestions. http: //wiki.apache.org/solr/SpellCheckComponent --> <searchComponent name= "spellcheck" class= "solr.SpellCheckComponent" > <str name= "queryAnalyzerFieldType" >text_general</str> <!-- Multiple "Spell Checkers" can be declared and used by this component --> <!-- a spellchecker built from a field of the main index --> <lst name= "spellchecker" > <str name= "name" > default </str> <str name= "field" >_text_</str> <str name= "classname" >solr.DirectSolrSpellChecker</str> <!-- the spellcheck distance measure used, the default is the internal levenshtein --> <str name= "distanceMeasure" >internal</str> <!-- minimum accuracy needed to be considered a valid spellcheck suggestion --> < float name= "accuracy" >0.5</ float > <!-- the maximum #edits we consider when enumerating terms: can be 1 or 2 --> < int name= "maxEdits" >2</ int > <!-- the minimum shared prefix when enumerating terms --> < int name= "minPrefix" >1</ int > <!-- maximum number of inspections per result. --> < int name= "maxInspections" >5</ int > <!-- minimum length of a query term to be considered for correction --> < int name= "minQueryLength" >4</ int > <!-- maximum threshold of documents a query term can appear to be considered for correction --> < float name= "maxQueryFrequency" >0.01</ float > <!-- uncomment this to require suggestions to occur in 1% of the documents < float name= "thresholdTokenFrequency" >.01</ float > --> </lst> <!-- a spellchecker that can break or combine words. See "/spell" handler below for usage --> <!-- <lst name= "spellchecker" > <str name= "name" >wordbreak</str> <str name= "classname" >solr.WordBreakSolrSpellChecker</str> <str name= "field" >name</str> <str name= "combineWords" > true </str> <str name= "breakWords" > true </str> < int name= "maxChanges" >10</ int > </lst> --> </searchComponent> <!-- A request handler for demonstrating the spellcheck component. NOTE: This is purely as an example. The whole purpose of the SpellCheckComponent is to hook it into the request handler that handles your normal user queries so that a separate request is not needed to get suggestions. IN OTHER WORDS, THERE IS REALLY GOOD CHANCE THE SETUP BELOW IS NOT WHAT YOU WANT FOR YOUR PRODUCTION SYSTEM! See http: //wiki.apache.org/solr/SpellCheckComponent for details on the request parameters. --> <requestHandler name= "/spell" class= "solr.SearchHandler" startup= "lazy" > <lst name= "defaults" > <!-- Solr will use suggestions from both the ' default ' spellchecker and from the 'wordbreak' spellchecker and combine them. collations (re-written queries) can include a combination of corrections from both spellcheckers --> <str name= "spellcheck.dictionary" > default </str> <str name= "spellcheck" >on</str> <str name= "spellcheck.extendedResults" > true </str> <str name= "spellcheck.count" >10</str> <str name= "spellcheck.alternativeTermCount" >5</str> <str name= "spellcheck.maxResultsForSuggest" >5</str> <str name= "spellcheck.collate" > true </str> <str name= "spellcheck.collateExtendedResults" > true </str> <str name= "spellcheck.maxCollationTries" >10</str> <str name= "spellcheck.maxCollations" >5</str> </lst> <arr name= "last-components" > <str>spellcheck</str> </arr> </requestHandler> <!-- Term Vector Component http: //wiki.apache.org/solr/TermVectorComponent --> <searchComponent name= "tvComponent" class= "solr.TermVectorComponent" /> <!-- A request handler for demonstrating the term vector component This is purely as an example. In reality you will likely want to add the component to your already specified request handlers. --> <requestHandler name= "/tvrh" class= "solr.SearchHandler" startup= "lazy" > <lst name= "defaults" > <bool name= "tv" > true </bool> </lst> <arr name= "last-components" > <str>tvComponent</str> </arr> </requestHandler> <!-- Clustering Component. (Omitted here. See the default Solr example for a typical configuration.) --> <!-- Terms Component http: //wiki.apache.org/solr/TermsComponent A component to return terms and document frequency of those terms --> <searchComponent name= "terms" class= "solr.TermsComponent" /> <!-- A request handler for demonstrating the terms component --> <requestHandler name= "/terms" class= "solr.SearchHandler" startup= "lazy" > <lst name= "defaults" > <bool name= "terms" > true </bool> <bool name= "distrib" > false </bool> </lst> <arr name= "components" > <str>terms</str> </arr> </requestHandler> <!-- Query Elevation Component http: //wiki.apache.org/solr/QueryElevationComponent a search component that enables you to configure the top results for a given query regardless of the normal lucene scoring. --> <searchComponent name= "elevator" class= "solr.QueryElevationComponent" > <!-- pick a fieldType to analyze queries --> <str name= "queryFieldType" >string</str> <str name= "config-file" >elevate.xml</str> </searchComponent> <!-- A request handler for demonstrating the elevator component --> <requestHandler name= "/elevate" class= "solr.SearchHandler" startup= "lazy" > <lst name= "defaults" > <str name= "echoParams" >explicit</str> </lst> <arr name= "last-components" > <str>elevator</str> </arr> </requestHandler> <!-- Highlighting Component http: //wiki.apache.org/solr/HighlightingParameters --> <searchComponent class= "solr.HighlightComponent" name= "highlight" > <highlighting> <!-- Configure the standard fragmenter --> <!-- This could most likely be commented out in the " default " case --> <fragmenter name= "gap" default = " true " class= "solr.highlight.GapFragmenter" > <lst name= "defaults" > < int name= "hl.fragsize" >100</ int > </lst> </fragmenter> <!-- A regular-expression-based fragmenter ( for sentence extraction) --> <fragmenter name= "regex" class= "solr.highlight.RegexFragmenter" > <lst name= "defaults" > <!-- slightly smaller fragsizes work better because of slop --> < int name= "hl.fragsize" >70</ int > <!-- allow 50% slop on fragment sizes --> < float name= "hl.regex.slop" >0.5</ float > <!-- a basic sentence pattern --> <str name= "hl.regex.pattern" >[-\w ,/\n\&quot;&apos;]{20,200}</str> </lst> </fragmenter> <!-- Configure the standard formatter --> <formatter name= "html" default = " true " class= "solr.highlight.HtmlFormatter" > <lst name= "defaults" > <str name= "hl.simple.pre" ><![CDATA[<em>]]></str> <str name= "hl.simple.post" ><![CDATA[</em>]]></str> </lst> </formatter> <!-- Configure the standard encoder --> <encoder name= "html" class= "solr.highlight.HtmlEncoder" /> <!-- Configure the standard fragListBuilder --> <fragListBuilder name= "simple" class= "solr.highlight.SimpleFragListBuilder" /> <!-- Configure the single fragListBuilder --> <fragListBuilder name= "single" class= "solr.highlight.SingleFragListBuilder" /> <!-- Configure the weighted fragListBuilder --> <fragListBuilder name= "weighted" default = " true " class= "solr.highlight.WeightedFragListBuilder" /> <!-- default tag FragmentsBuilder --> <fragmentsBuilder name= " default " default = " true " class= "solr.highlight.ScoreOrderFragmentsBuilder" > <!-- <lst name= "defaults" > <str name= "hl.multiValuedSeparatorChar" >/</str> </lst> --> </fragmentsBuilder> <!-- multi-colored tag FragmentsBuilder --> <fragmentsBuilder name= "colored" class= "solr.highlight.ScoreOrderFragmentsBuilder" > <lst name= "defaults" > <str name= "hl.tag.pre" ><![CDATA[ <b style= "background:yellow" >,<b style= "background:lawgreen" >, <b style= "background:aquamarine" >,<b style= "background:magenta" >, <b style= "background:palegreen" >,<b style= "background:coral" >, <b style= "background:wheat" >,<b style= "background:khaki" >, <b style= "background:lime" >,<b style= "background:deepskyblue" >]]></str> <str name= "hl.tag.post" ><![CDATA[</b>]]></str> </lst> </fragmentsBuilder> <boundaryScanner name= " default " default = " true " class= "solr.highlight.SimpleBoundaryScanner" > <lst name= "defaults" > <str name= "hl.bs.maxScan" >10</str> <str name= "hl.bs.chars" >.,!? &#9;&#10;&#13;</str> </lst> </boundaryScanner> <boundaryScanner name= "breakIterator" class= "solr.highlight.BreakIteratorBoundaryScanner" > <lst name= "defaults" > <!-- type should be one of CHARACTER, WORD( default ), LINE and SENTENCE --> <str name= "hl.bs.type" >WORD</str> <!-- language and country are used when constructing Locale object. --> <!-- And the Locale object will be used when getting instance of BreakIterator --> <str name= "hl.bs.language" >en</str> <str name= "hl.bs.country" >US</str> </lst> </boundaryScanner> </highlighting> </searchComponent> <!-- Update Processors Chains of Update Processor Factories for dealing with Update Requests can be declared, and then used by name in Update Request Processors http: //wiki.apache.org/solr/UpdateRequestProcessor --> <!-- Add unknown fields to the schema An example field type guessing update processor that will attempt to parse string-typed field values as Booleans, Longs, Doubles, or Dates, and then add schema fields with the guessed field types. This requires that the schema is both managed and mutable, by declaring schemaFactory as ManagedIndexSchemaFactory, with mutable specified as true . See http: //wiki.apache.org/solr/GuessingFieldTypes --> <updateRequestProcessorChain name= "add-unknown-fields-to-the-schema" > <!-- UUIDUpdateProcessorFactory will generate an id if none is present in the incoming document --> <processor class= "solr.UUIDUpdateProcessorFactory" /> <processor class= "solr.LogUpdateProcessorFactory" /> <processor class= "solr.DistributedUpdateProcessorFactory" /> <processor class= "solr.RemoveBlankFieldUpdateProcessorFactory" /> <processor class= "solr.FieldNameMutatingUpdateProcessorFactory" > <str name= "pattern" >[^\w-\.]</str> <str name= "replacement" >_</str> </processor> <processor class= "solr.ParseBooleanFieldUpdateProcessorFactory" /> <processor class= "solr.ParseLongFieldUpdateProcessorFactory" /> <processor class= "solr.ParseDoubleFieldUpdateProcessorFactory" /> <processor class= "solr.ParseDateFieldUpdateProcessorFactory" > <arr name= "format" > <str>yyyy-MM-dd'T'HH:mm:ss.SSSZ</str> <str>yyyy-MM-dd'T'HH:mm:ss,SSSZ</str> <str>yyyy-MM-dd'T'HH:mm:ss.SSS</str> <str>yyyy-MM-dd'T'HH:mm:ss,SSS</str> <str>yyyy-MM-dd'T'HH:mm:ssZ</str> <str>yyyy-MM-dd'T'HH:mm:ss</str> <str>yyyy-MM-dd'T'HH:mmZ</str> <str>yyyy-MM-dd'T'HH:mm</str> <str>yyyy-MM-dd HH:mm:ss.SSSZ</str> <str>yyyy-MM-dd HH:mm:ss,SSSZ</str> <str>yyyy-MM-dd HH:mm:ss.SSS</str> <str>yyyy-MM-dd HH:mm:ss,SSS</str> <str>yyyy-MM-dd HH:mm:ssZ</str> <str>yyyy-MM-dd HH:mm:ss</str> <str>yyyy-MM-dd HH:mmZ</str> <str>yyyy-MM-dd HH:mm</str> <str>yyyy-MM-dd</str> </arr> </processor> <processor class= "solr.AddSchemaFieldsUpdateProcessorFactory" > <str name= "defaultFieldType" >strings</str> <lst name= "typeMapping" > <str name= "valueClass" >java.lang. Boolean </str> <str name= "fieldType" >booleans</str> </lst> <lst name= "typeMapping" > <str name= "valueClass" >java.util.Date</str> <str name= "fieldType" >tdates</str> </lst> <lst name= "typeMapping" > <str name= "valueClass" >java.lang. Long </str> <str name= "valueClass" >java.lang. Integer </str> <str name= "fieldType" >tlongs</str> </lst> <lst name= "typeMapping" > <str name= "valueClass" >java.lang. Number </str> <str name= "fieldType" >tdoubles</str> </lst> </processor> <processor class= "solr.RunUpdateProcessorFactory" /> </updateRequestProcessorChain> <updateRequestProcessorChain name= "classification" > <processor class= "solr.ClassificationUpdateProcessorFactory" > <str name= "inputFields" >title_t^1.5,author_s</str> <str name= "classField" >cat_s</str> <str name= "algorithm" >bayes</str> </processor> </updateRequestProcessorChain> <!-- Deduplication An example dedup update processor that creates the "id" field on the fly based on the hash code of some other fields. This example has overwriteDupes set to false since we are using the id field as the signatureField and Solr will maintain uniqueness based on that anyway. --> <!-- <updateRequestProcessorChain name= "dedupe" > <processor class= "solr.processor.SignatureUpdateProcessorFactory" > <bool name= "enabled" > true </bool> <str name= "signatureField" >id</str> <bool name= "overwriteDupes" > false </bool> <str name= "fields" >name,features,cat</str> <str name= "signatureClass" >solr.processor.Lookup3Signature</str> </processor> <processor class= "solr.LogUpdateProcessorFactory" /> <processor class= "solr.RunUpdateProcessorFactory" /> </updateRequestProcessorChain> --> <!-- Language identification This example update chain identifies the language of the incoming documents using the langid contrib. The detected language is written to field language_s. No field name mapping is done. The fields used for detection are text, title, subject and description, making this example suitable for detecting languages form full-text rich documents injected via ExtractingRequestHandler. See more about langId at http: //wiki.apache.org/solr/LanguageDetection --> <!-- <updateRequestProcessorChain name= "langid" > <processor class= "org.apache.solr.update.processor.TikaLanguageIdentifierUpdateProcessorFactory" > <str name= "langid.fl" >text,title,subject,description</str> <str name= "langid.langField" >language_s</str> <str name= "langid.fallback" >en</str> </processor> <processor class= "solr.LogUpdateProcessorFactory" /> <processor class= "solr.RunUpdateProcessorFactory" /> </updateRequestProcessorChain> --> <!-- Script update processor This example hooks in an update processor implemented using JavaScript. See more about the script update processor at http: //wiki.apache.org/solr/ScriptUpdateProcessor --> <!-- <updateRequestProcessorChain name= "script" > <processor class= "solr.StatelessScriptUpdateProcessorFactory" > <str name= "script" >update-script.js</str> <lst name= "params" > <str name= "config_param" >example config parameter</str> </lst> </processor> <processor class= "solr.RunUpdateProcessorFactory" /> </updateRequestProcessorChain> --> <!-- Response Writers http: //wiki.apache.org/solr/QueryResponseWriter Request responses will be written using the writer specified by the 'wt' request parameter matching the name of a registered writer. The " default " writer is the default and will be used if 'wt' is not specified in the request. --> <!-- The following response writers are implicitly configured unless overridden... --> <!-- <queryResponseWriter name= "xml" default = " true " class= "solr.XMLResponseWriter" /> <queryResponseWriter name= "json" class= "solr.JSONResponseWriter" /> <queryResponseWriter name= "python" class= "solr.PythonResponseWriter" /> <queryResponseWriter name= "ruby" class= "solr.RubyResponseWriter" /> <queryResponseWriter name= "php" class= "solr.PHPResponseWriter" /> <queryResponseWriter name= "phps" class= "solr.PHPSerializedResponseWriter" /> <queryResponseWriter name= "csv" class= "solr.CSVResponseWriter" /> <queryResponseWriter name= "schema.xml" class= "solr.SchemaXmlResponseWriter" /> --> <queryResponseWriter name= "json" class= "solr.JSONResponseWriter" > <!-- For the purposes of the tutorial, JSON responses are written as plain text so that they are easy to read in *any* browser. If you expect a MIME type of "application/json" just remove this override. --> <str name= "content-type" >text/plain; charset=UTF-8</str> </queryResponseWriter> <!-- Custom response writers can be declared as needed... --> <queryResponseWriter name= "velocity" class= "solr.VelocityResponseWriter" startup= "lazy" > <str name= "template.base.dir" >${velocity.template.base.dir:}</str> <str name= "solr.resource.loader.enabled" >${velocity.solr.resource.loader.enabled: true }</str> <str name= "params.resource.loader.enabled" >${velocity.params.resource.loader.enabled: false }</str> </queryResponseWriter> <!-- XSLT response writer transforms the XML output by any xslt file found in Solr's conf/xslt directory. Changes to xslt files are checked for every xsltCacheLifetimeSeconds. --> <queryResponseWriter name= "xslt" class= "solr.XSLTResponseWriter" > < int name= "xsltCacheLifetimeSeconds" >5</ int > </queryResponseWriter> <!-- Query Parsers https: //cwiki.apache.org/confluence/display/solr/Query+Syntax+and+Parsing Multiple QParserPlugins can be registered by name, and then used in either the "defType" param for the QueryComponent (used by SearchHandler) or in LocalParams --> <!-- example of registering a query parser --> <!-- <queryParser name= "myparser" class= "com.mycompany.MyQParserPlugin" /> --> <!-- Function Parsers http: //wiki.apache.org/solr/FunctionQuery Multiple ValueSourceParsers can be registered by name, and then used as function names when using the "func" QParser. --> <!-- example of registering a custom function parser --> <!-- <valueSourceParser name= "myfunc" class= "com.mycompany.MyValueSourceParser" /> --> <!-- Document Transformers http: //wiki.apache.org/solr/DocTransformers --> <!-- Could be something like: <transformer name= "db" class= "com.mycompany.LoadFromDatabaseTransformer" > < int name= "connection" >jdbc: //....</ int > </transformer> To add a constant value to all docs, use: <transformer name= "mytrans2" class= "org.apache.solr.response.transform.ValueAugmenterFactory" > < int name= "value" >5</ int > </transformer> If you want the user to still be able to change it with _value:something_ use this : <transformer name= "mytrans3" class= "org.apache.solr.response.transform.ValueAugmenterFactory" > < double name= "defaultValue" >5</ double > </transformer> If you are using the QueryElevationComponent, you may wish to mark documents that get boosted. The EditorialMarkerFactory will do exactly that: <transformer name= "qecBooster" class= "org.apache.solr.response.transform.EditorialMarkerFactory" /> --> </config>
          Hide
          jpountz Adrien Grand added a comment -

          Yes it is included, the reason why it is not in the changelog is that the commit 15fdfd3caa54c1e6d8cf3435ded285c3b65bea8c on branch_6x did not add anything to solr/CHANGES.txt.

          Show
          jpountz Adrien Grand added a comment - Yes it is included, the reason why it is not in the changelog is that the commit 15fdfd3caa54c1e6d8cf3435ded285c3b65bea8c on branch_6x did not add anything to solr/CHANGES.txt.
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          Thanks Adrien for the prompt response!
          I was thinking the release note was generated automatically from Jira, but now it is clear.
          I guess now we are in a sort of inconsistency and adding to the solr/CHANGES.txt will not help.

          What about the cwiki ? Should we add the committer involved in that, in the conversation ?
          I created the documentation draft in the old wiki, but I have no priviledge to add it to the new cwiki.

          Cheers

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - Thanks Adrien for the prompt response! I was thinking the release note was generated automatically from Jira, but now it is clear. I guess now we are in a sort of inconsistency and adding to the solr/CHANGES.txt will not help. What about the cwiki ? Should we add the committer involved in that, in the conversation ? I created the documentation draft in the old wiki, but I have no priviledge to add it to the new cwiki. Cheers
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          Hi Tomas,
          as specified , I would prefer to move this to the Solr-user mailing list.
          This ticket was for the development, not for the usage

          Anyway I saw that you are using :

          <initParams path="/update/**">
          <lst name="defaults">
          <str name="update.chain">add-unknown-fields-to-the-schema</str>
          </lst>
          </initParams>

          Which is not using the classification UpdateRequestProcessor.

          Cheers

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - Hi Tomas, as specified , I would prefer to move this to the Solr-user mailing list. This ticket was for the development, not for the usage Anyway I saw that you are using : <initParams path="/update/**"> <lst name="defaults"> <str name="update.chain">add-unknown-fields-to-the-schema</str> </lst> </initParams> Which is not using the classification UpdateRequestProcessor. Cheers
          Hide
          jpountz Adrien Grand added a comment -

          I guess now we are in a sort of inconsistency and adding to the solr/CHANGES.txt will not help.

          I can add it now, this way a user upgrading to 6.2 will see that it was added in 6.1. I'll do it now.

          What about the cwiki ? Should we add the committer involved in that, in the conversation ?

          Hmm good question, I am not too familiar with the cwiki. Maybe someone else can help here?

          Show
          jpountz Adrien Grand added a comment - I guess now we are in a sort of inconsistency and adding to the solr/CHANGES.txt will not help. I can add it now, this way a user upgrading to 6.2 will see that it was added in 6.1. I'll do it now. What about the cwiki ? Should we add the committer involved in that, in the conversation ? Hmm good question, I am not too familiar with the cwiki. Maybe someone else can help here?
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 85020a555cb3fd934d3123f48dcd57e5f7679c88 in lucene-solr's branch refs/heads/branch_6_1 from Adrien Grand
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=85020a5 ]

          SOLR-7739: Add CHANGES entry.

          Show
          jira-bot ASF subversion and git services added a comment - Commit 85020a555cb3fd934d3123f48dcd57e5f7679c88 in lucene-solr's branch refs/heads/branch_6_1 from Adrien Grand [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=85020a5 ] SOLR-7739 : Add CHANGES entry.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit aaddefe80e950433a0d615bcc743552ab6079f5c in lucene-solr's branch refs/heads/branch_6x from Adrien Grand
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=aaddefe ]

          SOLR-7739: Add CHANGES entry.

          Show
          jira-bot ASF subversion and git services added a comment - Commit aaddefe80e950433a0d615bcc743552ab6079f5c in lucene-solr's branch refs/heads/branch_6x from Adrien Grand [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=aaddefe ] SOLR-7739 : Add CHANGES entry.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 97b6c5b8fd581f53f8d8d3fa74cde50ab6df7484 in lucene-solr's branch refs/heads/master from Adrien Grand
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=97b6c5b ]

          SOLR-7739: Add CHANGES entry.

          Show
          jira-bot ASF subversion and git services added a comment - Commit 97b6c5b8fd581f53f8d8d3fa74cde50ab6df7484 in lucene-solr's branch refs/heads/master from Adrien Grand [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=97b6c5b ] SOLR-7739 : Add CHANGES entry.
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          Thanks again Adrien!
          it sounds good !

          For the wiki let's try with Cassandra Targett, can you help us ?

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - Thanks again Adrien! it sounds good ! For the wiki let's try with Cassandra Targett , can you help us ?
          Hide
          tomas0413 Tomas Ramanauskas added a comment -

          Thanks, Alessandro. I fixed the config and I think the code for this feature is now being executed, however it's still not working. So I'll send an e-mail to Solr-user mailing list.

          Show
          tomas0413 Tomas Ramanauskas added a comment - Thanks, Alessandro. I fixed the config and I think the code for this feature is now being executed, however it's still not working. So I'll send an e-mail to Solr-user mailing list.
          Hide
          ctargett Cassandra Targett added a comment -

          We can add this to the Ref Guide, but it's too late for the 6.1 version. The existing documentation to be added to the Solr Ref Guide is in the above-referenced blog post? Even though it's too late for 6.1, I can put this in the draft area to be included for the next release in 6.2; people would be able to access it online until formal publication in the PDF.

          Show
          ctargett Cassandra Targett added a comment - We can add this to the Ref Guide, but it's too late for the 6.1 version. The existing documentation to be added to the Solr Ref Guide is in the above-referenced blog post? Even though it's too late for 6.1, I can put this in the draft area to be included for the next release in 6.2; people would be able to access it online until formal publication in the PDF.
          Hide
          alessandro.benedetti Alessandro Benedetti added a comment -

          Hi Cassandra Targett ,
          I already edited a little bit the documentation and published in here :

          https://wiki.apache.org/solr/SolrClassification

          Feel free to add any comment/ suggestion / editing !

          Cheers

          Show
          alessandro.benedetti Alessandro Benedetti added a comment - Hi Cassandra Targett , I already edited a little bit the documentation and published in here : https://wiki.apache.org/solr/SolrClassification Feel free to add any comment/ suggestion / editing ! Cheers
          Hide
          ctargett Cassandra Targett added a comment -

          I'm really far behind on this Alessandro Benedetti, my apologies. I added a reference to this URP to the Ref Guide page https://cwiki.apache.org/confluence/display/solr/Update+Request+Processors. None of the update processors are described very well there, so it would be strange to suddenly have a long section for just one, although we'd like someday to give full descriptions for all of them. For now, though, I linked to the Wiki page, and will work through some edits on that soon.

          Show
          ctargett Cassandra Targett added a comment - I'm really far behind on this Alessandro Benedetti , my apologies. I added a reference to this URP to the Ref Guide page https://cwiki.apache.org/confluence/display/solr/Update+Request+Processors . None of the update processors are described very well there, so it would be strange to suddenly have a long section for just one, although we'd like someday to give full descriptions for all of them. For now, though, I linked to the Wiki page, and will work through some edits on that soon.
          Hide
          sridhar2792 sridhar bandlamudi added a comment -

          hi Alessandro Benedetti
          i had gone through your post. have a question on indexing the training data into solr.
          can you give me simple demo on how to index the training documents.
          how should i label documents in to different classed and how can i make sure solr point to those labelled classes.
          should i add field called classField in my schema.xml

          ps:i'm news to solr may be i'm dumb at understanding few things

          Show
          sridhar2792 sridhar bandlamudi added a comment - hi Alessandro Benedetti i had gone through your post. have a question on indexing the training data into solr. can you give me simple demo on how to index the training documents. how should i label documents in to different classed and how can i make sure solr point to those labelled classes. should i add field called classField in my schema.xml ps:i'm news to solr may be i'm dumb at understanding few things

            People

            • Assignee:
              teofili Tommaso Teofili
              Reporter:
              alessandro.benedetti Alessandro Benedetti
            • Votes:
              5 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development