Solr
  1. Solr
  2. SOLR-6775

Creating backup snapshot null pointer exception

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 4.10
    • Fix Version/s: 5.1, 6.0
    • Component/s: replication (java)
    • Labels:
    • Environment:

      Linux Server, Java version "1.7.0_21", Solr version 4.10.0

      Description

      I set up Solr Replication. I have one master on a server, one slave on another server. The replication of data appears functioning correctly. The issue is when the master SOLR tries to create a snapshot backup it gets a null pointer exception.

      org.apache.solr.handler.SnapShooter createSnapshot method calls org.apache.solr.handler.SnapPuller.delTree(snapShotDir); at line 162 and the exception happens within org.apache.solr.handler.SnapPuller at line 1026 because snapShotDir is null.
      Here is the actual log output:

      58319963 [qtp12610551-16] INFO org.apache.solr.core.SolrCore - newest commit generation = 349
      58319983 [Thread-19] INFO org.apache.solr.handler.SnapShooter - Creating backup snapshot...
      Exception in thread "Thread-19" java.lang.NullPointerException
      at org.apache.solr.handler.SnapPuller.delTree(SnapPuller.java:1026)
      at org.apache.solr.handler.SnapShooter.createSnapshot(SnapShooter.java:162)
      at org.apache.solr.handler.SnapShooter$1.run(SnapShooter.java:91)

      I may have missed how to set the directory in the documentation but I've looked around without much luck. I thought the process was to use the same directory as the index data for the snapshots. Is this a known issue with this release or am I missing how to set the value? If someone could tell me how to set snapshotdir or confirm that it is an issue and a different way of backing up the index is needed it would be much appreciated.

      1. SOLR-6775_test_fix.patch
        6 kB
        Varun Thacker
      2. SOLR-6775.patch
        20 kB
        Varun Thacker
      3. SOLR-6775.patch
        0.8 kB
        Varun Thacker

        Activity

        Hide
        Varun Thacker added a comment -

        Hi Ryan,

        Can you copy-paste your replication hander setting and the API call that you made. I will try to reproduce it.

        Show
        Varun Thacker added a comment - Hi Ryan, Can you copy-paste your replication hander setting and the API call that you made. I will try to reproduce it.
        Hide
        Ryan Hesson added a comment -

        Sure, I really appreciate the help. Please let me know if I can share any more information.

        Thank you for your time,

        Ryan

        This is my setting on the master Solr index:

        <requestHandler name="/replication" class="solr.ReplicationHandler" >
        <lst name="master">
        <str name="replicateAfter">commit</str>
        <str name="replicateAfter">startup</str>
        <str name="backupAfter">optimize</str>
        <str name="confFiles">schema.xml,stopwords.txt</str>
        </lst>
        <int name="maxNumberOfBackups">2</int>
        </requestHandler>

        This is my setting on the slave Solr index:

        <requestHandler name="/replication" class="solr.ReplicationHandler" >
        <lst name="slave">
        <str name="masterUrl">[Server URL]:8983/solr/[Core Name]</str>
        <str name="pollInterval">01:00:00</str>
        </lst>
        </requestHandler>

        Far as API call, I know I'm doing add/update documents to the core and then a commit. Here's additional log info if it helps:

        58316473 [qtp12610551-16] INFO org.apache.solr.update.processor.LogUpdateProcessor – [Core Name] webapp=/solr path=/update params=

        {wt=javabin&version=2} {add=[559991 (1485389611244978176)]} 0 14
        58316500 [qtp12610551-16] INFO org.apache.solr.update.processor.LogUpdateProcessor – [Core Name] webapp=/solr path=/update params={wt=javabin&version=2}

        {add=[560030 (1485389611278532608)]}

        0 9
        58316507 [qtp12610551-16] INFO org.apache.solr.update.processor.LogUpdateProcessor – [Core Name] webapp=/solr path=/update params=

        {wt=javabin&version=2} {add=[539417 (1485389611289018368)]} 0 4
        58316523 [qtp12610551-16] INFO org.apache.solr.update.processor.LogUpdateProcessor – [Core Name] webapp=/solr path=/update params={wt=javabin&version=2}

        {add=[568646 (1485389611304747008)]}

        0 5
        58316537 [qtp12610551-16] INFO org.apache.solr.update.processor.LogUpdateProcessor – [Core Name] webapp=/solr path=/update params=

        {wt=javabin&version=2}

        {add=[635394 (1485389611318378496)]}

        0 6
        58316677 [qtp12610551-16] INFO org.apache.solr.update.UpdateHandler – start commit

        {,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}

        58316714 [qtp12610551-16] INFO org.apache.solr.core.SolrCore – SolrDeletionPolicy.onCommit: commits: num=2
        commit

        {dir=NRTCachingDirectory(NIOFSDirectory@/index/solr-4.10.0/[Company Name]/[Core Name]/data/index lockFactory=NativeFSLockFactory@/index/solr-4.10.0/[Company Name]/[Core Name]/data/index; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_9n,generation=347}

        commit

        {dir=NRTCachingDirectory(NIOFSDirectory@/index/solr-4.10.0/[Company Name]/[Core Name]/data/index lockFactory=NativeFSLockFactory@/index/solr-4.10.0/[Company Name]/[Core Name]/data/index; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_9o,generation=348}
        58316714 [qtp12610551-16] INFO org.apache.solr.core.SolrCore – newest commit generation = 348
        58316717 [qtp12610551-16] INFO org.apache.solr.search.SolrIndexSearcher – Opening Searcher@1ff79df[Core Name] main
        58316721 [qtp12610551-16] INFO org.apache.solr.update.UpdateHandler – end_commit_flush
        58316721 [searcherExecutor-6-thread-1] INFO org.apache.solr.core.SolrCore – QuerySenderListener sending requests to Searcher@1ff79df[Core Name] main{StandardDirectoryReader(segments_9o:1720:nrt _ik(4.10.0):C90144/5:delGen=1 _il(4.10.0):C5)}
        58316721 [searcherExecutor-6-thread-1] INFO org.apache.solr.core.SolrCore – QuerySenderListener done.
        58316722 [searcherExecutor-6-thread-1] INFO org.apache.solr.core.SolrCore – [Core Name] Registered new searcher Searcher@1ff79df[Core Name] main{StandardDirectoryReader(segments_9o:1720:nrt _ik(4.10.0):C90144/5:delGen=1 _il(4.10.0):C5)}
        58316735 [qtp12610551-16] INFO org.apache.solr.update.processor.LogUpdateProcessor – [Core Name] webapp=/solr path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 58
        58316737 [qtp12610551-16] INFO org.apache.solr.update.UpdateHandler – start commit{,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
        58319963 [qtp12610551-16] INFO org.apache.solr.core.SolrCore – SolrDeletionPolicy.onCommit: commits: num=2
        commit{dir=NRTCachingDirectory(NIOFSDirectory@/index/solr-4.10.0/[Company Name]/[Core Name]/data/index lockFactory=NativeFSLockFactory@/index/solr-4.10.0/[Company Name]/[Core Name]/data/index; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_9o,generation=348}

        commit

        {dir=NRTCachingDirectory(NIOFSDirectory@/index/solr-4.10.0/[Company Name]/[Core Name]/data/index lockFactory=NativeFSLockFactory@/index/solr-4.10.0/[Company Name]/[Core Name]/data/index; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_9p,generation=349}

        58319963 [qtp12610551-16] INFO org.apache.solr.core.SolrCore – newest commit generation = 349
        58319983 [Thread-19] INFO org.apache.solr.handler.SnapShooter – Creating backup snapshot...
        Exception in thread "Thread-19" java.lang.NullPointerException
        at org.apache.solr.handler.SnapPuller.delTree(SnapPuller.java:1026)
        at org.apache.solr.handler.SnapShooter.createSnapshot(SnapShooter.java:162)
        at org.apache.solr.handler.SnapShooter$1.run(SnapShooter.java:91)
        58320080 [qtp12610551-16] INFO org.apache.solr.search.SolrIndexSearcher – Opening Searcher@ff73af[Core Name] main
        58320083 [qtp12610551-16] INFO org.apache.solr.update.UpdateHandler – end_commit_flush
        58320083 [searcherExecutor-6-thread-1] INFO org.apache.solr.core.SolrCore – QuerySenderListener sending requests to Searcher@ff73af[Core Name] main

        {StandardDirectoryReader(segments_9p:1722:nrt _im(4.10.0):C90144)}

        58320084 [searcherExecutor-6-thread-1] INFO org.apache.solr.core.SolrCore – QuerySenderListener done.
        58320084 [searcherExecutor-6-thread-1] INFO org.apache.solr.core.SolrCore – [Core Name] Registered new searcher Searcher@ff73af[Core Name] main

        {StandardDirectoryReader(segments_9p:1722:nrt _im(4.10.0):C90144)}
        Show
        Ryan Hesson added a comment - Sure, I really appreciate the help. Please let me know if I can share any more information. Thank you for your time, Ryan This is my setting on the master Solr index: <requestHandler name="/replication" class="solr.ReplicationHandler" > <lst name="master"> <str name="replicateAfter">commit</str> <str name="replicateAfter">startup</str> <str name="backupAfter">optimize</str> <str name="confFiles">schema.xml,stopwords.txt</str> </lst> <int name="maxNumberOfBackups">2</int> </requestHandler> This is my setting on the slave Solr index: <requestHandler name="/replication" class="solr.ReplicationHandler" > <lst name="slave"> <str name="masterUrl"> [Server URL] :8983/solr/ [Core Name] </str> <str name="pollInterval">01:00:00</str> </lst> </requestHandler> Far as API call, I know I'm doing add/update documents to the core and then a commit. Here's additional log info if it helps: 58316473 [qtp12610551-16] INFO org.apache.solr.update.processor.LogUpdateProcessor – [Core Name] webapp=/solr path=/update params= {wt=javabin&version=2} {add=[559991 (1485389611244978176)]} 0 14 58316500 [qtp12610551-16] INFO org.apache.solr.update.processor.LogUpdateProcessor – [Core Name] webapp=/solr path=/update params={wt=javabin&version=2} {add=[560030 (1485389611278532608)]} 0 9 58316507 [qtp12610551-16] INFO org.apache.solr.update.processor.LogUpdateProcessor – [Core Name] webapp=/solr path=/update params= {wt=javabin&version=2} {add=[539417 (1485389611289018368)]} 0 4 58316523 [qtp12610551-16] INFO org.apache.solr.update.processor.LogUpdateProcessor – [Core Name] webapp=/solr path=/update params={wt=javabin&version=2} {add=[568646 (1485389611304747008)]} 0 5 58316537 [qtp12610551-16] INFO org.apache.solr.update.processor.LogUpdateProcessor – [Core Name] webapp=/solr path=/update params= {wt=javabin&version=2} {add=[635394 (1485389611318378496)]} 0 6 58316677 [qtp12610551-16] INFO org.apache.solr.update.UpdateHandler – start commit {,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false} 58316714 [qtp12610551-16] INFO org.apache.solr.core.SolrCore – SolrDeletionPolicy.onCommit: commits: num=2 commit {dir=NRTCachingDirectory(NIOFSDirectory@/index/solr-4.10.0/[Company Name]/[Core Name]/data/index lockFactory=NativeFSLockFactory@/index/solr-4.10.0/[Company Name]/[Core Name]/data/index; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_9n,generation=347} commit {dir=NRTCachingDirectory(NIOFSDirectory@/index/solr-4.10.0/[Company Name]/[Core Name]/data/index lockFactory=NativeFSLockFactory@/index/solr-4.10.0/[Company Name]/[Core Name]/data/index; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_9o,generation=348} 58316714 [qtp12610551-16] INFO org.apache.solr.core.SolrCore – newest commit generation = 348 58316717 [qtp12610551-16] INFO org.apache.solr.search.SolrIndexSearcher – Opening Searcher@1ff79df [Core Name] main 58316721 [qtp12610551-16] INFO org.apache.solr.update.UpdateHandler – end_commit_flush 58316721 [searcherExecutor-6-thread-1] INFO org.apache.solr.core.SolrCore – QuerySenderListener sending requests to Searcher@1ff79df [Core Name] main{StandardDirectoryReader(segments_9o:1720:nrt _ik(4.10.0):C90144/5:delGen=1 _il(4.10.0):C5)} 58316721 [searcherExecutor-6-thread-1] INFO org.apache.solr.core.SolrCore – QuerySenderListener done. 58316722 [searcherExecutor-6-thread-1] INFO org.apache.solr.core.SolrCore – [Core Name] Registered new searcher Searcher@1ff79df [Core Name] main{StandardDirectoryReader(segments_9o:1720:nrt _ik(4.10.0):C90144/5:delGen=1 _il(4.10.0):C5)} 58316735 [qtp12610551-16] INFO org.apache.solr.update.processor.LogUpdateProcessor – [Core Name] webapp=/solr path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 58 58316737 [qtp12610551-16] INFO org.apache.solr.update.UpdateHandler – start commit{,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false} 58319963 [qtp12610551-16] INFO org.apache.solr.core.SolrCore – SolrDeletionPolicy.onCommit: commits: num=2 commit{dir=NRTCachingDirectory(NIOFSDirectory@/index/solr-4.10.0/[Company Name]/[Core Name]/data/index lockFactory=NativeFSLockFactory@/index/solr-4.10.0/[Company Name]/[Core Name]/data/index; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_9o,generation=348} commit {dir=NRTCachingDirectory(NIOFSDirectory@/index/solr-4.10.0/[Company Name]/[Core Name]/data/index lockFactory=NativeFSLockFactory@/index/solr-4.10.0/[Company Name]/[Core Name]/data/index; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_9p,generation=349} 58319963 [qtp12610551-16] INFO org.apache.solr.core.SolrCore – newest commit generation = 349 58319983 [Thread-19] INFO org.apache.solr.handler.SnapShooter – Creating backup snapshot... Exception in thread "Thread-19" java.lang.NullPointerException at org.apache.solr.handler.SnapPuller.delTree(SnapPuller.java:1026) at org.apache.solr.handler.SnapShooter.createSnapshot(SnapShooter.java:162) at org.apache.solr.handler.SnapShooter$1.run(SnapShooter.java:91) 58320080 [qtp12610551-16] INFO org.apache.solr.search.SolrIndexSearcher – Opening Searcher@ff73af [Core Name] main 58320083 [qtp12610551-16] INFO org.apache.solr.update.UpdateHandler – end_commit_flush 58320083 [searcherExecutor-6-thread-1] INFO org.apache.solr.core.SolrCore – QuerySenderListener sending requests to Searcher@ff73af [Core Name] main {StandardDirectoryReader(segments_9p:1722:nrt _im(4.10.0):C90144)} 58320084 [searcherExecutor-6-thread-1] INFO org.apache.solr.core.SolrCore – QuerySenderListener done. 58320084 [searcherExecutor-6-thread-1] INFO org.apache.solr.core.SolrCore – [Core Name] Registered new searcher Searcher@ff73af [Core Name] main {StandardDirectoryReader(segments_9p:1722:nrt _im(4.10.0):C90144)}
        Hide
        Ryan Hesson added a comment -

        Not sure if this helps at all but I get the same NullPointerException using SOLR version 4.10.2 . Reading the documentation it sounds like SOLR is supposed to place snapshot backups in the same data directory as the index. I'm not sure how others would not experience this same scenario if they were to set up backupAfter in the ReplicationHandler.

        If you look at the code repository here for version 4.9:
        http://grepcode.com/file/repo1.maven.org/maven2/org.apache.solr/solr-core/4.9.0/org/apache/solr/handler/SnapShooter.java?av=f

        it doesn't appear the createSnapshot(...) method sets variable snapShotDir equal to anything before it sends it to the SnapPuller.delTree(snapShotDir) method. This appears to be the cause of the null pointer exception to me.

        Hope this helps,

        Ryan


        Ryan E. Hesson

        Show
        Ryan Hesson added a comment - Not sure if this helps at all but I get the same NullPointerException using SOLR version 4.10.2 . Reading the documentation it sounds like SOLR is supposed to place snapshot backups in the same data directory as the index. I'm not sure how others would not experience this same scenario if they were to set up backupAfter in the ReplicationHandler. If you look at the code repository here for version 4.9: http://grepcode.com/file/repo1.maven.org/maven2/org.apache.solr/solr-core/4.9.0/org/apache/solr/handler/SnapShooter.java?av=f it doesn't appear the createSnapshot(...) method sets variable snapShotDir equal to anything before it sends it to the SnapPuller.delTree(snapShotDir) method. This appears to be the cause of the null pointer exception to me. Hope this helps, Ryan – Ryan E. Hesson
        Hide
        Steve Fatula added a comment -

        I have the exact same issue on Solr 4.10.3.

        803906 [Thread-52] INFO org.apache.solr.handler.SnapShooter – Creating backup snapshot...
        Exception in thread "Thread-52" java.lang.NullPointerException
        at org.apache.solr.handler.SnapPuller.delTree(SnapPuller.java:1026)
        at org.apache.solr.handler.SnapShooter.createSnapshot(SnapShooter.java:162)
        at org.apache.solr.handler.SnapShooter$1.run(SnapShooter.java:91)

        Show
        Steve Fatula added a comment - I have the exact same issue on Solr 4.10.3. 803906 [Thread-52] INFO org.apache.solr.handler.SnapShooter – Creating backup snapshot... Exception in thread "Thread-52" java.lang.NullPointerException at org.apache.solr.handler.SnapPuller.delTree(SnapPuller.java:1026) at org.apache.solr.handler.SnapShooter.createSnapshot(SnapShooter.java:162) at org.apache.solr.handler.SnapShooter$1.run(SnapShooter.java:91)
        Hide
        Varun Thacker added a comment -

        Whoops thats my fault. Looks like when we split the duties and added a blocking call to validateCreateSnapshot , we forgot to call it when snapshots are triggered on commits.

        It was only added to explicit calls made to the snapshot command.

        So the fix is very simple, but I'll try improving the tests so that it can catch it in the future.

        Show
        Varun Thacker added a comment - Whoops thats my fault. Looks like when we split the duties and added a blocking call to validateCreateSnapshot , we forgot to call it when snapshots are triggered on commits. It was only added to explicit calls made to the snapshot command. So the fix is very simple, but I'll try improving the tests so that it can catch it in the future.
        Hide
        Ryan Hesson added a comment -

        Varun,

        I appreciate the update. I'm eager to set this up again.

        Can I ask what is the best way to path SOLR? Would we drop the SOLR-6775.patch file into a folder? Or if I download 4.10.2 again and reinstall will it now include the patch? I appreciate any suggestions.

        Thank you,

        Ryan


        Ryan E. Hesson
        Boscov's Department Stores, LLC
        Email: rhesson@boscovs.com
        Phone: (610)898-6185

        Show
        Ryan Hesson added a comment - Varun, I appreciate the update. I'm eager to set this up again. Can I ask what is the best way to path SOLR? Would we drop the SOLR-6775 .patch file into a folder? Or if I download 4.10.2 again and reinstall will it now include the patch? I appreciate any suggestions. Thank you, Ryan – Ryan E. Hesson Boscov's Department Stores, LLC Email: rhesson@boscovs.com Phone: (610)898-6185
        Hide
        Varun Thacker added a comment -
        • I added a test TestReplicationHandlerBackup.testBackupOnCommit which will fail without the fix in ReplicationHandler.
        • Cleaned up a bit of code in TestReplicationHandlerBackup.doTestBackup
        • Moved out CheckBackupStatus and reused it in BasicDistributedZk2Test.

        All tests pass and precommit is happy.

        Precommit was only happy after I did svn propset svn:eol-style native ./solr/core/src/test/org/apache/solr/handler/CheckBackupStatus.java

        Show
        Varun Thacker added a comment - I added a test TestReplicationHandlerBackup.testBackupOnCommit which will fail without the fix in ReplicationHandler. Cleaned up a bit of code in TestReplicationHandlerBackup.doTestBackup Moved out CheckBackupStatus and reused it in BasicDistributedZk2Test. All tests pass and precommit is happy. Precommit was only happy after I did svn propset svn:eol-style native ./solr/core/src/test/org/apache/solr/handler/CheckBackupStatus.java
        Hide
        Varun Thacker added a comment -

        Hi Ryan Hesson ,

        If you want to get it up and running before it comes out as part of an official release you could do this -

        1. Download the Solr 4.10 branch from here - https://github.com/apache/lucene-solr/tree/lucene_solr_4_10
        2. Apply the patch using patch -p0 < SOLR-6775.patch
        3. Then run 'ant example' from the solr/ directory. Once that is complete the solr war will be present under the dist folder. You could use that.

        Show
        Varun Thacker added a comment - Hi Ryan Hesson , If you want to get it up and running before it comes out as part of an official release you could do this - 1. Download the Solr 4.10 branch from here - https://github.com/apache/lucene-solr/tree/lucene_solr_4_10 2. Apply the patch using patch -p0 < SOLR-6775 .patch 3. Then run 'ant example' from the solr/ directory. Once that is complete the solr war will be present under the dist folder. You could use that.
        Hide
        ASF subversion and git services added a comment -

        Commit 1657523 from shalin@apache.org in branch 'dev/trunk'
        [ https://svn.apache.org/r1657523 ]

        SOLR-6775: Creating backup snapshot results in null pointer exception

        Show
        ASF subversion and git services added a comment - Commit 1657523 from shalin@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1657523 ] SOLR-6775 : Creating backup snapshot results in null pointer exception
        Hide
        ASF subversion and git services added a comment -

        Commit 1657524 from shalin@apache.org in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1657524 ]

        SOLR-6775: Creating backup snapshot results in null pointer exception

        Show
        ASF subversion and git services added a comment - Commit 1657524 from shalin@apache.org in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1657524 ] SOLR-6775 : Creating backup snapshot results in null pointer exception
        Hide
        Shalin Shekhar Mangar added a comment -

        Thanks Ryan and Varun!

        Show
        Shalin Shekhar Mangar added a comment - Thanks Ryan and Varun!
        Hide
        ASF subversion and git services added a comment -

        Commit 1657680 from shalin@apache.org in branch 'dev/trunk'
        [ https://svn.apache.org/r1657680 ]

        SOLR-6775: Import the right Lists class

        Show
        ASF subversion and git services added a comment - Commit 1657680 from shalin@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1657680 ] SOLR-6775 : Import the right Lists class
        Hide
        ASF subversion and git services added a comment -

        Commit 1657681 from shalin@apache.org in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1657681 ]

        SOLR-6775: Import the right Lists class

        Show
        ASF subversion and git services added a comment - Commit 1657681 from shalin@apache.org in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1657681 ] SOLR-6775 : Import the right Lists class
        Hide
        Mark Miller added a comment -

        The related tests have been failing on windows runs since this went in. A couple unrelated tests as well.

        junit.framework.TestSuite.org.apache.solr.core.TestSolrConfigHandler
        junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZk2Test
        org.apache.solr.handler.TestReplicationHandlerBackup.doTestBackup
        org.apache.solr.handler.TestReplicationHandlerBackup.testBackupOnCommit

        Show
        Mark Miller added a comment - The related tests have been failing on windows runs since this went in. A couple unrelated tests as well. junit.framework.TestSuite.org.apache.solr.core.TestSolrConfigHandler junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZk2Test org.apache.solr.handler.TestReplicationHandlerBackup.doTestBackup org.apache.solr.handler.TestReplicationHandlerBackup.testBackupOnCommit
        Hide
        Varun Thacker added a comment -

        I'll look into it.

        Show
        Varun Thacker added a comment - I'll look into it.
        Hide
        Mark Miller added a comment -

        I think a lot of tests even not on windows started failing with this...

        Show
        Mark Miller added a comment - I think a lot of tests even not on windows started failing with this...
        Hide
        Varun Thacker added a comment -

        Looking at one of the Jenkins here are the stack traces -

        [junit4] ERROR   3.52s | TestReplicationHandlerBackup.doTestBackup <<<
           [junit4]    > Throwable #1: java.io.IOException: Could not remove the following files (in the order of attempts):
           [junit4]    >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup E514A9DF49C1148E-001\solr-instance-001\collection1: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup E514A9DF49C1148E-001\solr-instance-001\collection1
           [junit4]    >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup E514A9DF49C1148E-001\solr-instance-001: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup E514A9DF49C1148E-001\solr-instance-001
           [junit4]    >        at __randomizedtesting.SeedInfo.seed([E514A9DF49C1148E:A49F89BA6E7FE7C1]:0)
           [junit4]    >        at org.apache.lucene.util.IOUtils.rm(IOUtils.java:294)
           [junit4]    >        at org.apache.solr.handler.TestReplicationHandler$SolrInstance.tearDown(TestReplicationHandler.java:1509)
           [junit4]    >        at org.apache.solr.handler.TestReplicationHandlerBackup.tearDown(TestReplicationHandlerBackup.java:115)
           [junit4]    >        at java.lang.Thread.run(Thread.java:745)
        

        and

        junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestReplicationHandlerBackup -Dtests.method=testBackupOnCommit -Dtests.seed=E514A9DF49C1148E -Dtests.slow=true -Dtests.locale=pt_PT -Dtests.timezone=Africa/Lubumbashi -Dtests.asserts=true -Dtests.file.encoding=UTF-8
           [junit4] ERROR   3.18s | TestReplicationHandlerBackup.testBackupOnCommit <<<
           [junit4]    > Throwable #1: java.io.IOException: Could not remove the following files (in the order of attempts):
           [junit4]    >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup E514A9DF49C1148E-001\solr-instance-002\collection1: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup E514A9DF49C1148E-001\solr-instance-002\collection1
           [junit4]    >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup E514A9DF49C1148E-001\solr-instance-002: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup E514A9DF49C1148E-001\solr-instance-002
           [junit4]    >        at __randomizedtesting.SeedInfo.seed([E514A9DF49C1148E:CCFC47343639A897]:0)
           [junit4]    >        at org.apache.lucene.util.IOUtils.rm(IOUtils.java:294)
           [junit4]    >        at org.apache.solr.handler.TestReplicationHandler$SolrInstance.tearDown(TestReplicationHandler.java:1509)
           [junit4]    >        at org.apache.solr.handler.TestReplicationHandlerBackup.tearDown(TestReplicationHandlerBackup.java:115)
        

        It happened because we added this to the solrconfig files - <str name="backupAfter">commit</str> which means an extra backup got created which we forgot to remove. I'll put up a patch to fix it.

        One question though - Why doesn't it reproduce on my mac?

        Show
        Varun Thacker added a comment - Looking at one of the Jenkins here are the stack traces - [junit4] ERROR 3.52s | TestReplicationHandlerBackup.doTestBackup <<< [junit4] > Throwable #1: java.io.IOException: Could not remove the following files (in the order of attempts): [junit4] > C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup E514A9DF49C1148E-001\solr-instance-001\collection1: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup E514A9DF49C1148E-001\solr-instance-001\collection1 [junit4] > C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup E514A9DF49C1148E-001\solr-instance-001: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup E514A9DF49C1148E-001\solr-instance-001 [junit4] > at __randomizedtesting.SeedInfo.seed([E514A9DF49C1148E:A49F89BA6E7FE7C1]:0) [junit4] > at org.apache.lucene.util.IOUtils.rm(IOUtils.java:294) [junit4] > at org.apache.solr.handler.TestReplicationHandler$SolrInstance.tearDown(TestReplicationHandler.java:1509) [junit4] > at org.apache.solr.handler.TestReplicationHandlerBackup.tearDown(TestReplicationHandlerBackup.java:115) [junit4] > at java.lang. Thread .run( Thread .java:745) and junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestReplicationHandlerBackup -Dtests.method=testBackupOnCommit -Dtests.seed=E514A9DF49C1148E -Dtests.slow= true -Dtests.locale=pt_PT -Dtests.timezone=Africa/Lubumbashi -Dtests.asserts= true -Dtests.file.encoding=UTF-8 [junit4] ERROR 3.18s | TestReplicationHandlerBackup.testBackupOnCommit <<< [junit4] > Throwable #1: java.io.IOException: Could not remove the following files (in the order of attempts): [junit4] > C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup E514A9DF49C1148E-001\solr-instance-002\collection1: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup E514A9DF49C1148E-001\solr-instance-002\collection1 [junit4] > C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup E514A9DF49C1148E-001\solr-instance-002: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup E514A9DF49C1148E-001\solr-instance-002 [junit4] > at __randomizedtesting.SeedInfo.seed([E514A9DF49C1148E:CCFC47343639A897]:0) [junit4] > at org.apache.lucene.util.IOUtils.rm(IOUtils.java:294) [junit4] > at org.apache.solr.handler.TestReplicationHandler$SolrInstance.tearDown(TestReplicationHandler.java:1509) [junit4] > at org.apache.solr.handler.TestReplicationHandlerBackup.tearDown(TestReplicationHandlerBackup.java:115) It happened because we added this to the solrconfig files - <str name="backupAfter">commit</str> which means an extra backup got created which we forgot to remove. I'll put up a patch to fix it. One question though - Why doesn't it reproduce on my mac?
        Hide
        Varun Thacker added a comment -

        Patch which doesn't attempt to cleanup the temp directory since LuceneTestCase.createTempDir states that "The folder will be automatically removed after the test class completes successfully."

        Show
        Varun Thacker added a comment - Patch which doesn't attempt to cleanup the temp directory since LuceneTestCase.createTempDir states that "The folder will be automatically removed after the test class completes successfully."
        Hide
        Shalin Shekhar Mangar added a comment -

        Thanks Varun. I committed your patch r1659149 on trunk and r1659151 on branch_5x.

        Show
        Shalin Shekhar Mangar added a comment - Thanks Varun. I committed your patch r1659149 on trunk and r1659151 on branch_5x.
        Hide
        ASF subversion and git services added a comment -

        Commit 1659149 from shalin@apache.org in branch 'dev/trunk'
        [ https://svn.apache.org/r1659149 ]

        SOLR-6775: Do not attempt cleanup of temp directory because it is handled by test framework

        Show
        ASF subversion and git services added a comment - Commit 1659149 from shalin@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1659149 ] SOLR-6775 : Do not attempt cleanup of temp directory because it is handled by test framework
        Hide
        ASF subversion and git services added a comment -

        Commit 1659151 from shalin@apache.org in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1659151 ]

        SOLR-6775: Do not attempt cleanup of temp directory because it is handled by test framework

        Show
        ASF subversion and git services added a comment - Commit 1659151 from shalin@apache.org in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1659151 ] SOLR-6775 : Do not attempt cleanup of temp directory because it is handled by test framework
        Hide
        Varun Thacker added a comment -

        Thanks Shalin! I'll keep an eye out on the builds

        Show
        Varun Thacker added a comment - Thanks Shalin! I'll keep an eye out on the builds
        Hide
        Timothy Potter added a comment -

        Bulk close after 5.1 release

        Show
        Timothy Potter added a comment - Bulk close after 5.1 release

          People

          • Assignee:
            Shalin Shekhar Mangar
            Reporter:
            Ryan Hesson
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development