Solr
  1. Solr
  2. SOLR-4801

"Can't find (or read) directory to add to classloader" at startup

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Not A Problem
    • Affects Version/s: 4.3
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Environment:

      Windows 2008 R2
      Java JRE 1.6.0_45 x64
      Jetty 8.1.10
      Solr 4.3.0 + lib\etc files

      Description

      I upgraded from 4.2.1 by replacing the war file and adding the lib\etc files from the example section into my Jetty installation. Now every time I start the services I get WARN messages I've never gotten before during startup.

      Additionally there appears to be profanity in the last WARN message as it looks for a "/total/crap/dir/ignored" after failing the other items.

      WARN  - 2013-05-08 08:46:06.710; org.apache.solr.core.SolrResourceLoader; Can't find (or read) directory to add to classloader: ../../../contrib/extraction/lib (resolved as: C:\Jetty\solr\instance1\..\..\..\contrib\extraction\lib).
      WARN  - 2013-05-08 08:46:06.714; org.apache.solr.core.SolrResourceLoader; Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: C:\Jetty\solr\instance1\..\..\..\dist).
      WARN  - 2013-05-08 08:46:06.715; org.apache.solr.core.SolrResourceLoader; Can't find (or read) directory to add to classloader: ../../../contrib/clustering/lib/ (resolved as: C:\Jetty\solr\instance1\..\..\..\contrib\clustering\lib).
      WARN  - 2013-05-08 08:46:06.716; org.apache.solr.core.SolrResourceLoader; Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: C:\Jetty\solr\instance1\..\..\..\dist).
      WARN  - 2013-05-08 08:46:06.716; org.apache.solr.core.SolrResourceLoader; Can't find (or read) directory to add to classloader: ../../../contrib/langid/lib/ (resolved as: C:\Jetty\solr\instance1\..\..\..\contrib\langid\lib).
      WARN  - 2013-05-08 08:46:06.717; org.apache.solr.core.SolrResourceLoader; Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: C:\Jetty\solr\instance1\..\..\..\dist).
      WARN  - 2013-05-08 08:46:06.717; org.apache.solr.core.SolrResourceLoader; Can't find (or read) directory to add to classloader: ../../../contrib/velocity/lib (resolved as: C:\Jetty\solr\instance1\..\..\..\contrib\velocity\lib).
      WARN  - 2013-05-08 08:46:06.718; org.apache.solr.core.SolrResourceLoader; Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: C:\Jetty\solr\instance1\..\..\..\dist).
      WARN  - 2013-05-08 08:46:06.719; org.apache.solr.core.SolrResourceLoader; Can't find (or read) directory to add to classloader: /total/crap/dir/ignored (resolved as: C:\Jetty\solr\instance1\total\crap\dir\ignored).
      

        Issue Links

          Activity

          Hide
          Aman Tandon added a comment - - edited

          Hi there i just started working on Solr from last few months.
          working environment: Windows/UNIX(Centos)

          It was easier to solve this problem just change the path in your solrconfig.xml like "D:\solr-tomcat\Apache Software Foundation\Tomcat 8.0\" (upto your dist folder in tomcat directory) if you don't have dist in tomcat then the dist folder from solr distribution and keep it parallel to tomcat and put path up to that directory

          here is the snapshot of my solrconfig.xml
          <lib dir="D:\solr-tomcat\Apache Software Foundation\Tomcat 8.0\contrib\extraction\lib" regex=".*\.jar" />
          <lib dir="D:\solr-tomcat\Apache Software Foundation\Tomcat 8.0\dist\" regex="solr-cell-\d.*\.jar" />

          <lib dir="D:\solr-tomcat\Apache Software Foundation\Tomcat 8.0\contrib\clustering\lib\" regex=".*\.jar" />
          <lib dir="D:\solr-tomcat\Apache Software Foundation\Tomcat 8.0\dist\" regex="solr-clustering-\d.*\.jar" />

          <lib dir="D:\solr-tomcat\Apache Software Foundation\Tomcat 8.0\contrib\langid\lib\" regex=".*\.jar" />
          <lib dir="D:\solr-tomcat\Apache Software Foundation\Tomcat 8.0\dist\" regex="solr-langid-\d.*\.jar" />

          <lib dir="D:\solr-tomcat\Apache Software Foundation\Tomcat 8.0\contrib\velocity\lib" regex=".*\.jar" />
          <lib dir="D:\solr-tomcat\Apache Software Foundation\Tomcat 8.0\dist\" regex="solr-velocity-\d.*\.jar" />

          Note: Use windows based directory separator the you will see no warning messages in logs after this correction

          Correct me if i am wrong.

          Thanks

          Show
          Aman Tandon added a comment - - edited Hi there i just started working on Solr from last few months. working environment: Windows/UNIX(Centos) It was easier to solve this problem just change the path in your solrconfig.xml like "D:\solr-tomcat\Apache Software Foundation\Tomcat 8.0\" (upto your dist folder in tomcat directory) if you don't have dist in tomcat then the dist folder from solr distribution and keep it parallel to tomcat and put path up to that directory here is the snapshot of my solrconfig.xml <lib dir="D:\solr-tomcat\Apache Software Foundation\Tomcat 8.0\contrib\extraction\lib" regex=".*\.jar" /> <lib dir="D:\solr-tomcat\Apache Software Foundation\Tomcat 8.0\dist\" regex="solr-cell-\d.*\.jar" /> <lib dir="D:\solr-tomcat\Apache Software Foundation\Tomcat 8.0\contrib\clustering\lib\" regex=".*\.jar" /> <lib dir="D:\solr-tomcat\Apache Software Foundation\Tomcat 8.0\dist\" regex="solr-clustering-\d.*\.jar" /> <lib dir="D:\solr-tomcat\Apache Software Foundation\Tomcat 8.0\contrib\langid\lib\" regex=".*\.jar" /> <lib dir="D:\solr-tomcat\Apache Software Foundation\Tomcat 8.0\dist\" regex="solr-langid-\d.*\.jar" /> <lib dir="D:\solr-tomcat\Apache Software Foundation\Tomcat 8.0\contrib\velocity\lib" regex=".*\.jar" /> <lib dir="D:\solr-tomcat\Apache Software Foundation\Tomcat 8.0\dist\" regex="solr-velocity-\d.*\.jar" /> Note: Use windows based directory separator the you will see no warning messages in logs after this correction Correct me if i am wrong. Thanks
          Hide
          Hoss Man added a comment -

          working as designed, see linked issue.

          Show
          Hoss Man added a comment - working as designed, see linked issue.
          Hide
          Shawn Heisey added a comment -

          I'd actually prefer if it was not "str", but rather custom tags. But in the new solr.xml format introduced in 4.3, it appears that the generic NamedList format was chosen for this.

          As you might have guessed, I had not looked into the new core discovery config. Once you mentioned NamedList, everything sorta clicked - that probably makes writing the code REALLY easy, which is a reasonable goal. As long as the syntax checking is good and will throw errors when something is mistyped, having code that's easy to write and understand is all good.

          Show
          Shawn Heisey added a comment - I'd actually prefer if it was not "str", but rather custom tags. But in the new solr.xml format introduced in 4.3, it appears that the generic NamedList format was chosen for this. As you might have guessed, I had not looked into the new core discovery config. Once you mentioned NamedList, everything sorta clicked - that probably makes writing the code REALLY easy, which is a reasonable goal. As long as the syntax checking is good and will throw errors when something is mistyped, having code that's easy to write and understand is all good.
          Hide
          Jan Høydahl added a comment -

          I hope that your proposed config example was just a quick brain dump and you don't really want "str" tags in solr.xml.

          Shawn Heisey I'd actually prefer if it was not "str", but rather custom tags. But in the new solr.xml format introduced in 4.3, it appears that the generic NamedList format was chosen for this.

          Show
          Jan Høydahl added a comment - I hope that your proposed config example was just a quick brain dump and you don't really want "str" tags in solr.xml. Shawn Heisey I'd actually prefer if it was not "str", but rather custom tags. But in the new solr.xml format introduced in 4.3, it appears that the generic NamedList format was chosen for this.
          Hide
          Leith Tussing added a comment -

          I dug into all of the solrconfig.xml files and removed the lines causing the warnings. Post service restart the logs are now empty of any WARNs at startup now.

          Show
          Leith Tussing added a comment - I dug into all of the solrconfig.xml files and removed the lines causing the warnings. Post service restart the logs are now empty of any WARNs at startup now.
          Hide
          Hoss Man added a comment -

          These warnings are being logged because of the changes made in SOLR-4653.

          In old versions of solr, non existent directories were happily ignored, as of SOLR-4653 these cause WARNing log messages.

          the decision was made in SOLR-4653 not to make this fatal however for reasons demonstrated by this bug report: people frequently copy/move/modify the example configs to new paths, where the relative includes of contribs used in the example no longer work – we didn't want those configs (that would have previously worked fine for people who didn't care about those lazy loaded contribs) to suddenly stop working.

          In short, from what i can tell, everything is working exactly as intended: solr is now warning the user that they have lib paths specified in their configs that can't be found – so thye either need to fix those paths, or delete them from thier configs, or ignore the warning messages.

          if the lib paths in question are strictly necessary for functionality to work (ie: a non lazy loaded request handler, or some other plugin) then the subsequent attempt to use/init that plugin would cause a hard failure, and the warning message would be available to help the user understand why the plugin failed to load

          Show
          Hoss Man added a comment - These warnings are being logged because of the changes made in SOLR-4653 . In old versions of solr, non existent directories were happily ignored, as of SOLR-4653 these cause WARNing log messages. the decision was made in SOLR-4653 not to make this fatal however for reasons demonstrated by this bug report: people frequently copy/move/modify the example configs to new paths, where the relative includes of contribs used in the example no longer work – we didn't want those configs (that would have previously worked fine for people who didn't care about those lazy loaded contribs) to suddenly stop working. In short, from what i can tell, everything is working exactly as intended: solr is now warning the user that they have lib paths specified in their configs that can't be found – so thye either need to fix those paths, or delete them from thier configs, or ignore the warning messages. if the lib paths in question are strictly necessary for functionality to work (ie: a non lazy loaded request handler, or some other plugin) then the subsequent attempt to use/init that plugin would cause a hard failure, and the warning message would be available to help the user understand why the plugin failed to load
          Hide
          Shawn Heisey added a comment -

          As far as I know we do not need the contrib items, however if I've incorrectly installed Solr I'd like to know how to fix it.

          In my mind, your minimalist install approach is the right way to go. I also take a minimalist approach to forking the example solrconfig.xml. When forking the example schema.xml, my approach is semi-minimalist - the "primitive" fieldType entries are very useful to keep around.

          If one of these lib directories does not exist, solr should fail hard. This total/crap/ignored is an anti-feature.

          As much as the little bit of profanity amuses me, I agree with Robert. Config elements that reference invalid information or have typos should result in a launch failure. For solrconfig.xml, that would result in that core not loading. The same goes for errors in a <core> tag in solr.xml. For parts of solr.xml outside of the <core> tags, it should result in Solr failing to start. Unrelated note: This is another indirect argument in favor of Solr no longer being a webapp.

          Jan Høydahl, I hope that your proposed config example was just a quick brain dump and you don't really want "str" tags in solr.xml. That would probably make it very hard to catch config errors at startup time.

          On a larger XML topic, using generic tags (str, int, etc) might make sense in parts of solrconfig.xml where it is used extensively, but I have sometimes wondered whether there would be fewer problems if we used more specific tags and fewer generic. Making it through the transition period might be a nightmare, though.

          Show
          Shawn Heisey added a comment - As far as I know we do not need the contrib items, however if I've incorrectly installed Solr I'd like to know how to fix it. In my mind, your minimalist install approach is the right way to go. I also take a minimalist approach to forking the example solrconfig.xml. When forking the example schema.xml, my approach is semi-minimalist - the "primitive" fieldType entries are very useful to keep around. If one of these lib directories does not exist, solr should fail hard. This total/crap/ignored is an anti-feature. As much as the little bit of profanity amuses me, I agree with Robert. Config elements that reference invalid information or have typos should result in a launch failure. For solrconfig.xml, that would result in that core not loading. The same goes for errors in a <core> tag in solr.xml. For parts of solr.xml outside of the <core> tags, it should result in Solr failing to start. Unrelated note: This is another indirect argument in favor of Solr no longer being a webapp. Jan Høydahl , I hope that your proposed config example was just a quick brain dump and you don't really want "str" tags in solr.xml. That would probably make it very hard to catch config errors at startup time. On a larger XML topic, using generic tags (str, int, etc) might make sense in parts of solrconfig.xml where it is used extensively, but I have sometimes wondered whether there would be fewer problems if we used more specific tags and fewer generic. Making it through the transition period might be a nightmare, though.
          Hide
          Robert Muir added a comment -

          Guys I seriously dont know if I should laugh or cry.

          If one of these lib directories does not exist, solr should fail hard. This total/crap/ignored is an anti-feature.

          Show
          Robert Muir added a comment - Guys I seriously dont know if I should laugh or cry. If one of these lib directories does not exist, solr should fail hard. This total/crap/ignored is an anti-feature.
          Hide
          Jan Høydahl added a comment -

          I vote for getting rid of all of these from the example solrconfig.xml and instead wire up the paths in solr.xml

          What about an option for recursive adding of shared libs in solr.xml, i.e. adding all jars from all sub folders?

            <str name="sharedLib" recurse="true">${solr.base.dir}/contrib</str>
            <str name="sharedLib" recurse="true">${solr.base.dir}/dist</str>
          

          We could then make the example Jetty automatically set/detect solr.base.dir, or if someone places them elsewhere, they can override it on command line.

          Show
          Jan Høydahl added a comment - I vote for getting rid of all of these from the example solrconfig.xml and instead wire up the paths in solr.xml What about an option for recursive adding of shared libs in solr.xml, i.e. adding all jars from all sub folders? <str name= "sharedLib" recurse= "true" > ${solr.base.dir}/contrib </str> <str name= "sharedLib" recurse= "true" > ${solr.base.dir}/dist </str> We could then make the example Jetty automatically set/detect solr.base.dir , or if someone places them elsewhere, they can override it on command line.
          Hide
          Leith Tussing added a comment -

          That's correct, I picked the war file and the lib\etc files from the package and put them into a non-example Jetty installation. I had copied over just the dist\solr-4.3.0.war and renamed it to solr.war at first which then resulted in failed loading. Looking at the logs informed me I was missing slf4j components so I then read the newly created http://wiki.apache.org/solr/SolrLogging which informed me to copy over the solr/example/lib/ext items as well.

          From everything I've testing the functionality is working as expected. I was able to add new entries and search existing ones. As far as I know we do not need the contrib items, however if I've incorrectly installed Solr I'd like to know how to fix it.

          I'm not exactly sure how to run the branch_4x (this correct? http://svn.apache.org/viewvc/lucene/dev/branches/branch_4x/solr/) or the normal 4.3.0 as source. If there is some guidance documentation you can point me to I would gladly give it a try.

          Show
          Leith Tussing added a comment - That's correct, I picked the war file and the lib\etc files from the package and put them into a non-example Jetty installation. I had copied over just the dist\solr-4.3.0.war and renamed it to solr.war at first which then resulted in failed loading. Looking at the logs informed me I was missing slf4j components so I then read the newly created http://wiki.apache.org/solr/SolrLogging which informed me to copy over the solr/example/lib/ext items as well. From everything I've testing the functionality is working as expected. I was able to add new entries and search existing ones. As far as I know we do not need the contrib items, however if I've incorrectly installed Solr I'd like to know how to fix it. I'm not exactly sure how to run the branch_4x (this correct? http://svn.apache.org/viewvc/lucene/dev/branches/branch_4x/solr/ ) or the normal 4.3.0 as source. If there is some guidance documentation you can point me to I would gladly give it a try.
          Hide
          Shawn Heisey added a comment -

          The profanity is in the log because it's in the example solrconfig.xml file, and this config was likely based on the example. We might want to replace that path in the example config with something less controversial. I find it amusing, and I hate giving into such pressures, but we have to live in the real world.

          I'm wondering if these warnings might be another symptom of SOLR-4791.

          Leith Tussing, does everything work despite the warnings? If you used the entire Solr download as-is, those paths should exist, but if you've copied only required bits from example into a separate Jetty installation, they would not exist. Do you need any of the contrib features in one of the directories that are mentioned? Is testing a branch_4x checkout something you can do? A more relevant option would be to try 4.3.0 source with the patch from SOLR-4791.

          Show
          Shawn Heisey added a comment - The profanity is in the log because it's in the example solrconfig.xml file, and this config was likely based on the example. We might want to replace that path in the example config with something less controversial. I find it amusing, and I hate giving into such pressures, but we have to live in the real world. I'm wondering if these warnings might be another symptom of SOLR-4791 . Leith Tussing , does everything work despite the warnings? If you used the entire Solr download as-is, those paths should exist, but if you've copied only required bits from example into a separate Jetty installation, they would not exist. Do you need any of the contrib features in one of the directories that are mentioned? Is testing a branch_4x checkout something you can do? A more relevant option would be to try 4.3.0 source with the patch from SOLR-4791 .
          Hide
          Mark Miller added a comment -

          I've mainly only heard of laughter as a response, so perhaps it's all balanced out in the world. I'm still ready to scold the culprit. I saw a guy end his interview on the BBC early by using such language

          I do think those warning messages are (kind of oddly) expected now unless you remove those lib directives from the solrconfig.xml.

          Show
          Mark Miller added a comment - I've mainly only heard of laughter as a response, so perhaps it's all balanced out in the world. I'm still ready to scold the culprit. I saw a guy end his interview on the BBC early by using such language I do think those warning messages are (kind of oddly) expected now unless you remove those lib directives from the solrconfig.xml.
          Hide
          Leith Tussing added a comment -

          The profanity part doesn't bother me, but there's probably someone out there who will bust a blood vessel if they saw it.

          Show
          Leith Tussing added a comment - The profanity part doesn't bother me, but there's probably someone out there who will bust a blood vessel if they saw it.
          Hide
          Mark Miller added a comment -

          Woah, that's some vulgar stuff Don't worry, I know who is responsible and they won't get away with it unscathed.

          Show
          Mark Miller added a comment - Woah, that's some vulgar stuff Don't worry, I know who is responsible and they won't get away with it unscathed.

            People

            • Assignee:
              Unassigned
              Reporter:
              Leith Tussing
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development