Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-8449

Multiple restores on the same core does not work

    Details

      Description

      Hi all, I facing the following issue with Solr 5.2.1 and the ongoing version 5.4.
      The restore functionality is not working under Linux and causing an exception on Windows machines each time you want to restore an existing backup twice or even more.

      Steps to reproduce:
      1. Start a Solr instance pointing the solr_home to e.g. the example-DIH folder.
      2. Select a core, e.g. the “solr” core.
      3. Switch to the “Documents” tab
      4. Add a document

      {“id”:”1”,”title”:”change.me”}

      5. Do a backup with the following API call “/solr/replication?command=backup&name=test”
      The backup defaults to the location solr_home/solr/data/snapshot.test
      6. Add a document to the index

      {“id”:”2”,”title”:”change.me”}

      . Now there a two document in the index.
      7. Restore the back with the following call “/solr/replication?command=restore&name=test”
      New index location “solr_home/solr/data/restore.snapshot.test” is created without any physical file in it, except the file write.lock. Num Docs is now 1 as expected!
      8. Add a document to the index

      {“id”:”3”,”title”:”change.me”}

      . Now there a two document in the index.
      9. Restore the same previous created back again with the following call “/solr/replication?command=restore&name=test”. Notice, there are still 2 docs in the index!!!
      10. Try to restore again, but still the same, 2 docs in the index…
      11. Shut down Solr, you will see the index folder “solr_home/solr/data/restore.snapshot.test” disappears.
      12. Restart Solr. You will notice the following log entry “Solr index directory ‘solr_home/solr/data/restore.snapshot.test’ doesn’t exist. Creating new index”, and indeed the Index is empty, showing 0 documents.
      13. After the restart, I tried to restore the existing backup again without any results…

      I thing this behavior in not intended!!!

      Even more Problems arise when you run Solr on a Windows machine.
      After step 10 a folder “index” is created under “solr_home/solr/data/” with a write.lock file in it. After that, the following exception is thrown:
      …Error closing IndexWriter
      java.lang.IllegalStateException: file: MMapDirectory@D:\solr\Solr_versions\solr-5.2.1\...restore.snapshot.test lockFactory=org.apache.lucene.store.Nat
      iveFSLockFactory@3d3d7a0e appears both in delegate and in cache

      The log file from the Windows test is attached.

      Let me know if you have problems reproducing the same behavior,

      Regards Johannes

      1. SOLR-8449.patch
        2 kB
        Varun Thacker
      2. SOLR-8449.patch
        4 kB
        Varun Thacker
      3. log_windows7_sp1.txt
        46 kB
        Johannes Brucher

        Activity

        Hide
        jomarley Johannes Brucher added a comment - - edited

        Exceptions from the windows 7 restore test attached...

        Show
        jomarley Johannes Brucher added a comment - - edited Exceptions from the windows 7 restore test attached...
        Hide
        varunthacker Varun Thacker added a comment -

        4. Add a document

        Do you call commit after that?

        Show
        varunthacker Varun Thacker added a comment - 4. Add a document Do you call commit after that?
        Hide
        jomarley Johannes Brucher added a comment - - edited

        I uploaded the document via the Admin UI/Core/Documents tab.
        I think using this tab will automatically perform a commit in the background I guess, because after adding the doc its in the index.

        But to answer your question, after step 4 i did not explicitly call a commit statment!

        Show
        jomarley Johannes Brucher added a comment - - edited I uploaded the document via the Admin UI/Core/Documents tab. I think using this tab will automatically perform a commit in the background I guess, because after adding the doc its in the index. But to answer your question, after step 4 i did not explicitly call a commit statment!
        Hide
        varunthacker Varun Thacker added a comment -

        Patch which fixes the problem.

        Previously we never supported restoring the same backup . Hence this bug was found.

        I've tried it manually and it works with the steps listed out here. Still need to write some tests before committing this

        Show
        varunthacker Varun Thacker added a comment - Patch which fixes the problem. Previously we never supported restoring the same backup . Hence this bug was found. I've tried it manually and it works with the steps listed out here. Still need to write some tests before committing this
        Hide
        varunthacker Varun Thacker added a comment -

        Patch which fixes the problem and updated test case which will fail without the fix .

        The patch is a low risk fix. So Nicholas Knize cool if we backport it to 6.0 as well?

        Show
        varunthacker Varun Thacker added a comment - Patch which fixes the problem and updated test case which will fail without the fix . The patch is a low risk fix. So Nicholas Knize cool if we backport it to 6.0 as well?
        Hide
        nknize Nicholas Knize added a comment -

        +1 for 6.0 backport

        Show
        nknize Nicholas Knize added a comment - +1 for 6.0 backport
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 4381018b77f019ce2c579240538177b546ff3d86 in lucene-solr's branch refs/heads/master from Varun Thacker
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4381018 ]

        SOLR-8449: Fix the core restore functionality to allow restoring multiple times on the same core

        Show
        jira-bot ASF subversion and git services added a comment - Commit 4381018b77f019ce2c579240538177b546ff3d86 in lucene-solr's branch refs/heads/master from Varun Thacker [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4381018 ] SOLR-8449 : Fix the core restore functionality to allow restoring multiple times on the same core
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit cb4a91eb7c61678939cd65d739ed57c4335181a3 in lucene-solr's branch refs/heads/branch_6x from Varun Thacker
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=cb4a91e ]

        SOLR-8449: Fix the core restore functionality to allow restoring multiple times on the same core

        Show
        jira-bot ASF subversion and git services added a comment - Commit cb4a91eb7c61678939cd65d739ed57c4335181a3 in lucene-solr's branch refs/heads/branch_6x from Varun Thacker [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=cb4a91e ] SOLR-8449 : Fix the core restore functionality to allow restoring multiple times on the same core
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit be021fcf6f3c87ab5ed51fb0f5dbfd08d752a899 in lucene-solr's branch refs/heads/branch_6_0 from Varun Thacker
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=be021fc ]

        SOLR-8449: Fix the core restore functionality to allow restoring multiple times on the same core

        Show
        jira-bot ASF subversion and git services added a comment - Commit be021fcf6f3c87ab5ed51fb0f5dbfd08d752a899 in lucene-solr's branch refs/heads/branch_6_0 from Varun Thacker [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=be021fc ] SOLR-8449 : Fix the core restore functionality to allow restoring multiple times on the same core
        Hide
        varunthacker Varun Thacker added a comment -

        Thanks Johannes! This is now fixed for 6.0

        Show
        varunthacker Varun Thacker added a comment - Thanks Johannes! This is now fixed for 6.0
        Hide
        anshumg Anshum Gupta added a comment - - edited

        back porting for 5.5.1

        Show
        anshumg Anshum Gupta added a comment - - edited back porting for 5.5.1
        Hide
        anshumg Anshum Gupta added a comment -

        branch_5x:

        commit 443fd2d29a4326a2d483c33bcbfeb7e2f636f250
        Author: Varun Thacker <varunthacker1989@gmail.com>
        Date:   Sat Mar 5 13:15:19 2016 +0530
        
            SOLR-8449: Fix the core restore functionality to allow restoring multiple times on the same core
        

        branch_5_5

        commit efb7d4463ee5f1146ee193a46f6a146ca3f48d67
        Author: Varun Thacker <varunthacker1989@gmail.com>
        Date:   Sat Mar 5 13:15:19 2016 +0530
        
            SOLR-8449: Fix the core restore functionality to allow restoring multiple times on the same core
        
        Show
        anshumg Anshum Gupta added a comment - branch_5x: commit 443fd2d29a4326a2d483c33bcbfeb7e2f636f250 Author: Varun Thacker <varunthacker1989@gmail.com> Date: Sat Mar 5 13:15:19 2016 +0530 SOLR-8449: Fix the core restore functionality to allow restoring multiple times on the same core branch_5_5 commit efb7d4463ee5f1146ee193a46f6a146ca3f48d67 Author: Varun Thacker <varunthacker1989@gmail.com> Date: Sat Mar 5 13:15:19 2016 +0530 SOLR-8449: Fix the core restore functionality to allow restoring multiple times on the same core

          People

          • Assignee:
            varunthacker Varun Thacker
            Reporter:
            jomarley Johannes Brucher
          • Votes:
            2 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development