Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-9120

LukeRequestHandler logs WARN "Error getting file length for [segments_NNN]" for inconsequential NoSuchFileException situations -- looks scary but is not a problem, logging should be reduced

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 5.5, 6.0
    • Fix Version/s: 7.2, master (8.0)
    • Component/s: None
    • Labels:
      None

      Description

      Begining with Solr 5.5, the LukeRequestHandler started attempting to report the name and file size of the segments file for the current Searcher+IndexReader in use by Solr – however the filesize information is not always available from the Directory in cases where "on disk" commits have caused that file to be removed, for example...

      • you perform index updates & commits w/o "newSearcher" being opened
      • you "concurrently" make requests to the LukeRequestHandler or the CoreAdminHandler requesting "STATUS" (ie: after the commit, before any newSearcher)
        • these requests can come from the Admin UI passively if it's open in a browser

      In situations like this, a decision was made in SOLR-8587 to log a WARNing in the event that the segments file size could not be determined – but these WARNing messages look scary and have lead (many) users to assume something is wrong with their solr index.

      We should reduce the severity of these log messages, and improve the wording to make it more clear that this is not a fundemental problem with the index.


      Here's some trivial steps to reproduce the WARN message...

      $ bin/solr -e techproducts
      ...
      $ tail -f example/techproducts/logs/solr.log
      ...
      

      In another terminal...

      $ curl -H 'Content-Type: application/json' 'http://localhost:8983/solr/techproducts/update?commit=true&openSearcher=false' --data-binary '[{"id":"HOSS"}]'
      ...
      $ curl 'http://localhost:8983/solr/techproducts/admin/luke'
      ...
      

      When the "/admin/luke" URL is hit, this will show up in the logs – but the luke request will finish correctly...

      WARN  - 2017-11-08 17:23:44.574; [   x:techproducts] org.apache.solr.handler.admin.LukeRequestHandler; Error getting file length for [segments_2]
      java.nio.file.NoSuchFileException: /home/hossman/lucene/dev/solr/example/techproducts/solr/techproducts/data/index/segments_2
      	at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
      	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
      	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
      	at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
      	at sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
      	at sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
      	at java.nio.file.Files.readAttributes(Files.java:1737)
      	at java.nio.file.Files.size(Files.java:2332)
      	at org.apache.lucene.store.FSDirectory.fileLength(FSDirectory.java:243)
      	at org.apache.lucene.store.NRTCachingDirectory.fileLength(NRTCachingDirectory.java:128)
      	at org.apache.solr.handler.admin.LukeRequestHandler.getFileLength(LukeRequestHandler.java:611)
      	at org.apache.solr.handler.admin.LukeRequestHandler.getIndexInfo(LukeRequestHandler.java:584)
      	at org.apache.solr.handler.admin.LukeRequestHandler.handleRequestBody(LukeRequestHandler.java:136)
      	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
      ...
      INFO  - 2017-11-08 17:23:44.587; [   x:techproducts] org.apache.solr.core.SolrCore; [techproducts]  webapp=/solr path=/admin/luke params={} status=0 QTime=15
      

        Attachments

        1. SOLR-9120.patch
          4 kB
          Shawn Heisey
        2. SOLR-9120.patch
          4 kB
          Shawn Heisey
        3. SOLR-9120.patch
          3 kB
          Hoss Man

          Issue Links

            Activity

              People

              • Assignee:
                hossman Hoss Man
                Reporter:
                markus17 Markus Jelsma
              • Votes:
                18 Vote for this issue
                Watchers:
                47 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: