Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-6367

empty tlog on HDFS when hard crash - no docs to replay on recovery

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments


    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: 5.0, 6.0
    • Component/s: None
    • Labels:


      Filing this bug based on an email to solr-user@lucene from Tom Chen (Fri, 18 Jul 2014)...

      Reproduce steps:
      1) Setup Solr to run on HDFS like this:

      java -Dsolr.directoryFactory=HdfsDirectoryFactory

      For the purpose of this testing, turn off the default auto commit in solrconfig.xml, i.e. comment out autoCommit like this:


      2) Add a document without commit:
      {{curl "http://localhost:8983/solr/collection1/update?commit=false" -H
      "Content-type:text/xml; charset=utf-8" --data-binary "@solr.xml"}}

      3) Solr generate empty tlog file (0 file size, the last one ends with 6):

      [hadoop@hdtest042 exampledocs]$ hadoop fs -ls
      Found 5 items
      -rw-r--r--   1 hadoop hadoop        667 2014-07-18 08:47
      -rw-r--r--   1 hadoop hadoop         67 2014-07-18 08:47
      -rw-r--r--   1 hadoop hadoop        667 2014-07-18 08:47
      -rw-r--r--   1 hadoop hadoop          0 2014-07-18 09:02
      -rw-r--r--   1 hadoop hadoop          0 2014-07-18 09:02

      4) Simulate Solr crash by killing the process with -9 option.

      5) restart the Solr process. Observation is that uncommitted document are
      not replayed, files in tlog directory are cleaned up. Hence uncommitted
      document(s) is lost.

      Am I missing anything or this is a bug?

      BTW, additional observations:
      a) If in step 4) Solr is stopped gracefully (i.e. without -9 option),
      non-empty tlog file is geneated and after re-starting Solr, uncommitted
      document is replayed as expected.

      b) If Solr doesn't run on HDFS (i.e. on local file system), this issue is
      not observed either.


        Issue Links



            • Assignee:
              markrmiller@gmail.com Mark Miller
              hossman Chris M. Hostetter


              • Created:

                Issue deployment