Solr
  1. Solr
  2. SOLR-2500

TestSolrProperties sometimes fails with "no such core: core0"

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 4.0-ALPHA
    • Fix Version/s: 3.2, 4.0-ALPHA
    • Component/s: None
    • Labels:
      None

      Description

      [junit] Testsuite: org.apache.solr.client.solrj.embedded.TestSolrProperties
      [junit] Testcase: testProperties(org.apache.solr.client.solrj.embedded.TestSolrProperties): Caused an ERROR
      [junit] No such core: core0
      [junit] org.apache.solr.common.SolrException: No such core: core0
      [junit] at org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:118)
      [junit] at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:105)
      [junit] at org.apache.solr.client.solrj.embedded.TestSolrProperties.testProperties(TestSolrProperties.java:128)
      [junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1260)
      [junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1189)

      1. SOLR-2500.patch
        3 kB
        Doron Cohen
      2. SOLR-2500.patch
        0.5 kB
        Robert Muir
      3. SOLR-2500.patch
        4 kB
        Robert Muir
      4. solr-after-1st-run.xml
        0.8 kB
        Doron Cohen
      5. solr-clean.xml
        2 kB
        Doron Cohen

        Activity

        Hide
        Uwe Schindler added a comment -

        Bulk cose after release of 3.2

        Show
        Uwe Schindler added a comment - Bulk cose after release of 3.2
        Hide
        Steve Rowe added a comment -

        I attached a patch with a fix to SOLR-2331, which introduced the problem.

        Show
        Steve Rowe added a comment - I attached a patch with a fix to SOLR-2331 , which introduced the problem.
        Hide
        Steve Rowe added a comment -

        seems like calling gc() is just masking the problem? we should hunt down which finalizer is closing the file and explicitly close instead / fix the leak?

        I agree.

        I tracked down the actual file activity to SolrXMLSerializer.persistFile() - this class was created as part of SOLR-2331, which Mark M. committed 2 days ago; the timing makes it the likely culprit.

        Show
        Steve Rowe added a comment - seems like calling gc() is just masking the problem? we should hunt down which finalizer is closing the file and explicitly close instead / fix the leak? I agree. I tracked down the actual file activity to SolrXMLSerializer.persistFile() - this class was created as part of SOLR-2331 , which Mark M. committed 2 days ago; the timing makes it the likely culprit.
        Hide
        Robert Muir added a comment -

        seems like calling gc() is just masking the problem? we should hunt down which finalizer is closing the file and explicitly close instead / fix the leak?

        Show
        Robert Muir added a comment - seems like calling gc() is just masking the problem? we should hunt down which finalizer is closing the file and explicitly close instead / fix the leak?
        Hide
        Steve Rowe added a comment -

        Reopening to address Windows test failure.

        Show
        Steve Rowe added a comment - Reopening to address Windows test failure.
        Hide
        Steve Rowe added a comment -

        I can get this failure to consistently succeed by calling System.gc() prior to the attempt to delete the file.

        Any objections to adding this?

        Show
        Steve Rowe added a comment - I can get this failure to consistently succeed by calling System.gc() prior to the attempt to delete the file. Any objections to adding this?
        Hide
        Steve Rowe added a comment -

        On Windows 7 using Oracle JDK 1.6.0_21, TestSolrProperties#testProperties() is consistently failing for me, both individually and with all Solr tests, and Ant and in IntelliJ:

        java.lang.AssertionError: Failed to delete C:\svn\lucene\dev\trunk\solr\build\tests\solr\shared\solr-persist.xml
        at org.junit.Assert.fail(Assert.java:91)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.client.solrj.embedded.TestSolrProperties.tearDown(TestSolrProperties.java:107)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
        at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
        at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1430)
        at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1348)
        at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
        at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
        at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
        at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:157)

        The failure is in TestSolrProperties.tearDown():

        107: assertTrue("Failed to delete "+persistedFile, persistedFile.delete());
        
        Show
        Steve Rowe added a comment - On Windows 7 using Oracle JDK 1.6.0_21, TestSolrProperties#testProperties() is consistently failing for me, both individually and with all Solr tests, and Ant and in IntelliJ: java.lang.AssertionError: Failed to delete C:\svn\lucene\dev\trunk\solr\build\tests\solr\shared\solr-persist.xml at org.junit.Assert.fail(Assert.java:91) at org.junit.Assert.assertTrue(Assert.java:43) at org.apache.solr.client.solrj.embedded.TestSolrProperties.tearDown(TestSolrProperties.java:107) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1430) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1348) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) at org.junit.runners.ParentRunner.run(ParentRunner.java:236) at org.junit.runner.JUnitCore.run(JUnitCore.java:157) The failure is in TestSolrProperties.tearDown(): 107: assertTrue( "Failed to delete " +persistedFile, persistedFile.delete());
        Hide
        Robert Muir added a comment -

        Bulk close for 3.2

        Show
        Robert Muir added a comment - Bulk close for 3.2
        Hide
        Doron Cohen added a comment -

        fixed in trunk: r1125932.
        merged to 3x: r1125942.

        Show
        Doron Cohen added a comment - fixed in trunk: r1125932. merged to 3x: r1125942.
        Hide
        Steve Rowe added a comment -

        fixed title

        Show
        Steve Rowe added a comment - fixed title
        Hide
        Robert Muir added a comment -

        +1 to commit! Thanks Doron!

        Show
        Robert Muir added a comment - +1 to commit! Thanks Doron!
        Hide
        Steve Rowe added a comment -

        I ran the TestSolrProperties test case alone, without the patch, from both Maven and IntelliJ, and I get the same behavior: it passes once, then fails on every successive attempt, unless "clean" is performed first.

        With the patch applied, the test passes under both IntelliJ and Maven, regardless of whether "clean" is performed first.

        Show
        Steve Rowe added a comment - I ran the TestSolrProperties test case alone, without the patch, from both Maven and IntelliJ, and I get the same behavior: it passes once, then fails on every successive attempt, unless "clean" is performed first. With the patch applied, the test passes under both IntelliJ and Maven, regardless of whether "clean" is performed first.
        Hide
        Doron Cohen added a comment -

        Attached patch, test passes now in both IDE and cmd line:

        • at setup() copies solr.xml to a private file.
        • use that private file as its solr.solr.home.
        • erase that file at tearDown(), though not erasing it
          should not affect on further/re/tests.
        • fixes the deletion at tearDown() to look at
          solr.solr.home rather than solr.home.
          (I think this was a bug on a bug in this test - it used the
          original file at s.s.h but for cleanup
          attempted to remove files from just s.h.

        This debugging took place in pure darkness, better review...

        Show
        Doron Cohen added a comment - Attached patch, test passes now in both IDE and cmd line: at setup() copies solr.xml to a private file. use that private file as its solr.solr.home. erase that file at tearDown(), though not erasing it should not affect on further/re/tests. fixes the deletion at tearDown() to look at solr.solr.home rather than solr.home. (I think this was a bug on a bug in this test - it used the original file at s.s.h but for cleanup attempted to remove files from just s.h. This debugging took place in pure darkness, better review...
        Hide
        Robert Muir added a comment -

        ok, so really the right way to fix this I think, is to ensure the test is only working from its private dir and copies stuff it needs in there.

        Then this will work fine from the IDE too (the patch only causes ant to recopy a clean version the next time you run 'test')

        Show
        Robert Muir added a comment - ok, so really the right way to fix this I think, is to ensure the test is only working from its private dir and copies stuff it needs in there. Then this will work fine from the IDE too (the patch only causes ant to recopy a clean version the next time you run 'test')
        Hide
        Michael McCandless added a comment -

        Fix works for me!!

        Show
        Michael McCandless added a comment - Fix works for me!!
        Hide
        Yonik Seeley added a comment - - edited

        I don't have much time to look at this right now (and I don't really know the test), but I just tried running it directly from intellij and that failed also.

        First, note that it tries to use something off the CWD... but then the core container is created under build/tests:

        INFO: pwd: /opt/code/lusolr/.
        2011-05-19 20.52.58 org.apache.solr.core.CoreContainer <init>
        INFO: New CoreContainer 746169063
        2011-05-19 20.52.58 org.apache.solr.core.SolrResourceLoader <init>
        INFO: Solr home set to '/opt/code/lusolr/solr/build/tests/solr/shared/'
        2011-05-19 20.52.58 org.apache.solr.core.SolrResourceLoader <init>
        

        This causes a problem when it comes time to delete later:

        WARNING: !!!! WARNING: best effort to remove /opt/code/lusolr/solr/shared/data FAILED !!!!!
        

        Of course the weird thing is that tearDown() only tries to delete the data directory and not the whole solr home... this seems incorrect?
        That would lead to leaving around an old solr.xml file (since it's outside the data directory) and could cause issues the next time the test is run.

        edit: crossed messages w/ robert above - looks like the issue has already been found+fixed.

        Show
        Yonik Seeley added a comment - - edited I don't have much time to look at this right now (and I don't really know the test), but I just tried running it directly from intellij and that failed also. First, note that it tries to use something off the CWD... but then the core container is created under build/tests: INFO: pwd: /opt/code/lusolr/. 2011-05-19 20.52.58 org.apache.solr.core.CoreContainer <init> INFO: New CoreContainer 746169063 2011-05-19 20.52.58 org.apache.solr.core.SolrResourceLoader <init> INFO: Solr home set to '/opt/code/lusolr/solr/build/tests/solr/shared/' 2011-05-19 20.52.58 org.apache.solr.core.SolrResourceLoader <init> This causes a problem when it comes time to delete later: WARNING: !!!! WARNING: best effort to remove /opt/code/lusolr/solr/shared/data FAILED !!!!! Of course the weird thing is that tearDown() only tries to delete the data directory and not the whole solr home... this seems incorrect? That would lead to leaving around an old solr.xml file (since it's outside the data directory) and could cause issues the next time the test is run. edit: crossed messages w/ robert above - looks like the issue has already been found+fixed.
        Hide
        Robert Muir added a comment -

        Its also worth mentioning this patch won't help eclipse at all, its only a workaround for ant at the moment.

        Show
        Robert Muir added a comment - Its also worth mentioning this patch won't help eclipse at all, its only a workaround for ant at the moment.
        Hide
        Robert Muir added a comment -

        The attached patch is a workaround for the issue for now, but we should fix the test to be "cleaner" as I don't like whats going on here.

        Whats happening is the test changes its solr.xml configuration file, which is in build/tests/solr/shared/solr.xml. The next time you run the tests, it wont copy over this file because it has a newer time.

        In my opinion the test should really make its own private home so it won't meddle with other tests or have problems like this (we can fix the test to do this), but this is a simple intermediate fix if you guys don't mind testing it.

        Show
        Robert Muir added a comment - The attached patch is a workaround for the issue for now, but we should fix the test to be "cleaner" as I don't like whats going on here. Whats happening is the test changes its solr.xml configuration file, which is in build/tests/solr/shared/solr.xml. The next time you run the tests, it wont copy over this file because it has a newer time. In my opinion the test should really make its own private home so it won't meddle with other tests or have problems like this (we can fix the test to do this), but this is a simple intermediate fix if you guys don't mind testing it.
        Hide
        Robert Muir added a comment -

        OK, i think you might be right... TestSolrProperties is the one that just failed for me.

        I'll look into this test now (though I'm still confused about TestSolrCoreProperties but i'll let that be)

        Show
        Robert Muir added a comment - OK, i think you might be right... TestSolrProperties is the one that just failed for me. I'll look into this test now (though I'm still confused about TestSolrCoreProperties but i'll let that be)
        Hide
        Michael McCandless added a comment -

        For me, it's TestSolrProperties that reliably fails it's it's been run before. Ie, it passes on first run after "ant clean" but then fails thereafter.

        TestSolrCoreProperties seems to run fine.

        (Fedora 13).

        Show
        Michael McCandless added a comment - For me, it's TestSolrProperties that reliably fails it's it's been run before. Ie, it passes on first run after "ant clean" but then fails thereafter. TestSolrCoreProperties seems to run fine. (Fedora 13).
        Hide
        Doron Cohen added a comment -

        Oops just noticed I was testing all this time TestSolrProperties and not TestSolrCoreProperties, and, because the error message was the same as in the issue description "No such core: core0" I was sure that this is the same test... Now this is confusing...

        Hmmm.. the original exception reported above is
        [junit] at org.apache.solr.client.solrj.embedded.TestSolrProperties.testProperties(TestSolrProperties.java:128)

        So perhaps I was working on the correct bug after all and just the JIRA issue title is inaccurate?
        Or I need to call it a day...

        Anyhow, TestSolrProperties consistently behaves as I described here, while TestSolrCoreProperties consistently passes (when ran in standalone mode).

        Show
        Doron Cohen added a comment - Oops just noticed I was testing all this time TestSolrProperties and not TestSolrCoreProperties, and, because the error message was the same as in the issue description "No such core: core0" I was sure that this is the same test... Now this is confusing... Hmmm.. the original exception reported above is [junit] at org.apache.solr.client.solrj.embedded.TestSolrProperties.testProperties(TestSolrProperties.java:128) So perhaps I was working on the correct bug after all and just the JIRA issue title is inaccurate? Or I need to call it a day... Anyhow, TestSolrProperties consistently behaves as I described here, while TestSolrCoreProperties consistently passes (when ran in standalone mode).
        Hide
        Doron Cohen added a comment -

        FWIW, also the first clean run would fail if test's tearDown() is modified like this:

        -    persistedFile.delete();
        +    assertTrue("could not delete "+persistedFile, persistedFile.delete());
        

        For some reason it fails to remove that file - in both Linux and Windows.

        Show
        Doron Cohen added a comment - FWIW, also the first clean run would fail if test's tearDown() is modified like this: - persistedFile.delete(); + assertTrue("could not delete "+persistedFile, persistedFile.delete()); For some reason it fails to remove that file - in both Linux and Windows.
        Hide
        Robert Muir added a comment -

        I guess the real question is: why doesn't the test work if rewritten like this?

        Bug in TestHarness?
        Bug in CoreContainer/properties loading functionality itself?

        Show
        Robert Muir added a comment - I guess the real question is: why doesn't the test work if rewritten like this? Bug in TestHarness? Bug in CoreContainer/properties loading functionality itself?
        Hide
        Doron Cohen added a comment -

        solr.xml files from trunk/bin/solr/shared:

        • clean - with which the test passes.
        • after-1st-run - with which it fails.
        Show
        Doron Cohen added a comment - solr.xml files from trunk/bin/solr/shared: clean - with which the test passes. after-1st-run - with which it fails.
        Hide
        Robert Muir added a comment -

        In Eclipse, after cleaning the project the test passes, and then start failing in all successive runs.

        FYI This is the behavior I've noticed when running the test from Ant also... a 'clean' seems to workaround the issue...

        Show
        Robert Muir added a comment - In Eclipse, after cleaning the project the test passes, and then start failing in all successive runs. FYI This is the behavior I've noticed when running the test from Ant also... a 'clean' seems to workaround the issue...
        Hide
        Doron Cohen added a comment -

        From Eclipse (XP), passed at 1st attempt, failed at the 2nd!

        I am not familiar with this part of the code so it would be too much work to track it all the way myself, but I think I can now provide sufficient information for solving it.

        In Eclipse, after cleaning the project the test passes, and then start failing in all successive runs.
        So I assume when you run it isolated you also do clean, which covers Eclipse's clean (and more).

        I tracked the content of the cleaned relevant dir before and after the test - it is (trunk/)bin/solr - there's only one file that differs between the runs - this is bin/solr/shared/solr.xml.

        Not sure if this is a bug in the test not cleaning after itself or a bug in the code that reads the configuration...

        I'll attach here the two file so that you can compare them.

        Show
        Doron Cohen added a comment - From Eclipse (XP), passed at 1st attempt, failed at the 2nd! I am not familiar with this part of the code so it would be too much work to track it all the way myself, but I think I can now provide sufficient information for solving it. In Eclipse, after cleaning the project the test passes, and then start failing in all successive runs. So I assume when you run it isolated you also do clean, which covers Eclipse's clean (and more). I tracked the content of the cleaned relevant dir before and after the test - it is (trunk/)bin/solr - there's only one file that differs between the runs - this is bin/solr/shared/solr.xml. Not sure if this is a bug in the test not cleaning after itself or a bug in the code that reads the configuration... I'll attach here the two file so that you can compare them.
        Hide
        Uwe Schindler added a comment -

        I get crazy with that test...

        It passes isolated, but fails on my machine every run together with other tests.

        Show
        Uwe Schindler added a comment - I get crazy with that test... It passes isolated, but fails on my machine every run together with other tests.

          People

          • Assignee:
            Doron Cohen
            Reporter:
            Robert Muir
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development