Solr
  1. Solr
  2. SOLR-1070

IndexSearcher not closed after indexing with DataImportHandler

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.4
    • Fix Version/s: 1.4
    • Component/s: clients - java
    • Labels:
      None
    • Environment:

      Debian 2.6.26-1-xen-amd64
      Tomcat 5.5

      Description

      What I have noticed is afecting to the nightly build using multicore. I am using tomcat 5.5 server and Debian 2.6.26-1-xen-amd64.
      After indexing and optimizing with DataImporthandler the old IndexSearcher is not being proper closed.
      To reproduce the bug I have done a full-import from a mysql database optimizing at the end. Once done I change the data from the database and index optimizing again.
      When now I do a lsof | grep solr I can see that tomcat is holding the deleted files from old index.
      I can see the error in the stats screen aswell... if I do 4 full-imports, stats will show me 3 opened IndexSearchers.
      As I index on the master and never use it to serve search requests I have sorted it modifying the function getSearcher in SolrCore.java
      There's a part in the source where is coded:

      // we are all done with the old searcher we used
      // for warming...
      if (currSearcherHolderF!=null)

      {currSearcherHolderF.decref();}

      I force there to close the indexSearcher and remvoe it from the RefCounted:

      // we are all done with the old searcher we used
      // for warming...
      if (currSearcherHolderF!=null)

      { currSearcherHolderF.get().close(); currSearcherHolderF.decref(); _searchers.remove(); }

      As Yonik told me:
      >> Forcing it to close at the point you did is unsafe since other threads
      >> may still be using that searcher.

      My solution is definitely not good if search requests must be served on that core of that server...

        Activity

        Hide
        Shalin Shekhar Mangar added a comment -

        I don't think this is related to DataImportHandler. DIH calls commit using a CommitUpdateCommand the same way as an xml commit request would.

        Show
        Shalin Shekhar Mangar added a comment - I don't think this is related to DataImportHandler. DIH calls commit using a CommitUpdateCommand the same way as an xml commit request would.
        Hide
        Marc Sturlese added a comment -

        Yes, I saw it uses CommitUpdateCommand, wich executes a commit from DirectUpdateHandler2.java. Haven't tried to index via xml with tomcat.
        Seems that some code is not decrementing the reference decref() from SolrIndexSearcher once it's done

        Show
        Marc Sturlese added a comment - Yes, I saw it uses CommitUpdateCommand, wich executes a commit from DirectUpdateHandler2.java. Haven't tried to index via xml with tomcat. Seems that some code is not decrementing the reference decref() from SolrIndexSearcher once it's done
        Hide
        Yonik Seeley added a comment -

        I haven't been able to reproduce any problem with normal XML update commands (with multicore either)... seems likely related to DIH somehow.

        Marc, what's the refCount on the core with too many searchers (check the admin stats page)? It should be "2"... one because the core is in service, and one for the request that is generating the stats page.

        Show
        Yonik Seeley added a comment - I haven't been able to reproduce any problem with normal XML update commands (with multicore either)... seems likely related to DIH somehow. Marc, what's the refCount on the core with too many searchers (check the admin stats page)? It should be "2"... one because the core is in service, and one for the request that is generating the stats page.
        Hide
        Noble Paul added a comment - - edited

        this is the code which does a commit in DIH

        //SolrWriter#commit()
         CommitUpdateCommand commit = new CommitUpdateCommand(optimize);
         updateProcessor.processCommit(commit);
        

        is there anything else we need to do to fix this

        DIH never uses an IndexSearcher. Do you have a Transformer or something which does a Context#SolrCore#getSearcher()

        Show
        Noble Paul added a comment - - edited this is the code which does a commit in DIH //SolrWriter#commit() CommitUpdateCommand commit = new CommitUpdateCommand(optimize); updateProcessor.processCommit(commit); is there anything else we need to do to fix this DIH never uses an IndexSearcher. Do you have a Transformer or something which does a Context#SolrCore#getSearcher()
        Hide
        Marc Sturlese added a comment -

        Hi Noble, I think you show me the light.
        Sorry for the loss of time, I think it was my fault.
        In the transformer, once the index is done I save the Index Version in a database doing:

        Context.getSolrCore().getSearcher().get().getReader().getVersion();

        I thought it was just returning me the version and not opening a new searcher. After doing that I wasn't closing or decrementing anything.
        So, can you give me an advice how to get the current searcher version in a correct way? I am not sure what should I decrement or close to avoid that problem...

        Thanks and sorry again.

        Show
        Marc Sturlese added a comment - Hi Noble, I think you show me the light. Sorry for the loss of time, I think it was my fault. In the transformer, once the index is done I save the Index Version in a database doing: Context.getSolrCore().getSearcher().get().getReader().getVersion(); I thought it was just returning me the version and not opening a new searcher. After doing that I wasn't closing or decrementing anything. So, can you give me an advice how to get the current searcher version in a correct way? I am not sure what should I decrement or close to avoid that problem... Thanks and sorry again.
        Hide
        Shalin Shekhar Mangar added a comment -

        Use it like this:

        RefCounted<SolrIndexSearcher> searchHolder = null;
        try {
          searchHolder = Context.getSolrCore().getSearcher();
          IndexReader reader = searchHolder.get().getReader();
          // Do whatever you need to            
        } finally {
          if (searchHolder != null) searchHolder.decref();
        }
        
        Show
        Shalin Shekhar Mangar added a comment - Use it like this: RefCounted<SolrIndexSearcher> searchHolder = null ; try { searchHolder = Context.getSolrCore().getSearcher(); IndexReader reader = searchHolder.get().getReader(); // Do whatever you need to } finally { if (searchHolder != null ) searchHolder.decref(); }

          People

          • Assignee:
            Unassigned
            Reporter:
            Marc Sturlese
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development