Solr
  1. Solr
  2. SOLR-5164

In some cases, creating collections via the Collections API due to core being created in the wrong directory

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 4.5, Trunk
    • Fix Version/s: 4.5, Trunk
    • Component/s: SolrCloud
    • Labels:
      None

      Description

      When you try to create a collection in SolrCloud, the instanceDir that gets created has an extra "solr" in it which messes up the pathing for all the <lib> directives in solrconfig.xml as they're all relative.

      1. SOLR-5164.patch
        2 kB
        Erick Erickson

        Issue Links

          Activity

          Hide
          Shawn Heisey added a comment - - edited

          In a cloud (or manually sharded) environment, IMHO it's better to put jars in $solrhome/lib because they'll be automatically loaded at startup and available to all cores.

          Because we can't assume symlink support, it's really difficult to use this method with the example.

          Show
          Shawn Heisey added a comment - - edited In a cloud (or manually sharded) environment, IMHO it's better to put jars in $solrhome/lib because they'll be automatically loaded at startup and available to all cores. Because we can't assume symlink support, it's really difficult to use this method with the example.
          Hide
          Erick Erickson added a comment -

          Alan Woodward Well, this has to be the most time I've spent changing about 4 lines.

          If you can please take a glance at it. These seem fairly safe changes, they both just bring back what was in the code a while ago. Besides the solr/solr issue, when creating a properties file in Cloud mode, it appears that the parent directory hasn't been created first, so that step was failing.

          If you don't apply it (and backport it to 4.4) I'll do it in the morning. But you have a 5 hour head start <G>...

          Show
          Erick Erickson added a comment - Alan Woodward Well, this has to be the most time I've spent changing about 4 lines. If you can please take a glance at it. These seem fairly safe changes, they both just bring back what was in the code a while ago. Besides the solr/solr issue, when creating a properties file in Cloud mode, it appears that the parent directory hasn't been created first, so that step was failing. If you don't apply it (and backport it to 4.4) I'll do it in the morning. But you have a 5 hour head start <G>...
          Hide
          Erick Erickson added a comment -

          Shawn Heisey It's not a matter of putting the jars in the right place at all. The core was just being created in the wrong place so the relative pathing from the stock solrconfig.xml couldn't work. The Carrotsearch classes that couldn't be found are part of the stock solrconfig distribution....

          Show
          Erick Erickson added a comment - Shawn Heisey It's not a matter of putting the jars in the right place at all. The core was just being created in the wrong place so the relative pathing from the stock solrconfig.xml couldn't work. The Carrotsearch classes that couldn't be found are part of the stock solrconfig distribution....
          Hide
          Alan Woodward added a comment -

          Related: SOLR-5099.

          I think we need an explicit test for creating collections via the API, though. It's a bit scary that this bug can occur without the test suite complaining about it. I'm busy for the next couple of days, but will have some time next week if nobody else gets there first.

          Show
          Alan Woodward added a comment - Related: SOLR-5099 . I think we need an explicit test for creating collections via the API, though. It's a bit scary that this bug can occur without the test suite complaining about it. I'm busy for the next couple of days, but will have some time next week if nobody else gets there first.
          Hide
          Erick Erickson added a comment -

          Blast, I wish I'd paid more attention to SOLR-5099, it'd have saved me some time. Sigh..

          Alan Woodward I looked and there are some collection creation tests, but I didn't dig enough to understand completely why the second solr in the path didn't trip this condition. What it didn't seem like we have was a way to restart from scratch. And in the case of SOLR-5099, core creation does succeed it's the restart that's the problem.

          FWIW.

          Show
          Erick Erickson added a comment - Blast, I wish I'd paid more attention to SOLR-5099 , it'd have saved me some time. Sigh.. Alan Woodward I looked and there are some collection creation tests, but I didn't dig enough to understand completely why the second solr in the path didn't trip this condition. What it didn't seem like we have was a way to restart from scratch. And in the case of SOLR-5099 , core creation does succeed it's the restart that's the problem. FWIW.
          Hide
          ASF subversion and git services added a comment -

          Commit 1514666 from Erick Erickson in branch 'dev/trunk'
          [ https://svn.apache.org/r1514666 ]

          SOLR-5164, Can not create a collection via collections API (cloud mode). Fixes SOLR-5099 too

          Show
          ASF subversion and git services added a comment - Commit 1514666 from Erick Erickson in branch 'dev/trunk' [ https://svn.apache.org/r1514666 ] SOLR-5164 , Can not create a collection via collections API (cloud mode). Fixes SOLR-5099 too
          Hide
          ASF subversion and git services added a comment -

          Commit 1514684 from Erick Erickson in branch 'dev/branches/branch_4x'
          [ https://svn.apache.org/r1514684 ]

          SOLR-5164, Can not create a collection via collections API (cloud mode). Fixes SOLR-5099 too

          Show
          ASF subversion and git services added a comment - Commit 1514684 from Erick Erickson in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1514684 ] SOLR-5164 , Can not create a collection via collections API (cloud mode). Fixes SOLR-5099 too
          Hide
          Mark Miller added a comment -

          We should add a test case to the collections api that catches this.

          Also, did this affect 4.4? The Affects versions seems to indicate not? If thats the case, there should be no separate changes entry.

          Show
          Mark Miller added a comment - We should add a test case to the collections api that catches this. Also, did this affect 4.4? The Affects versions seems to indicate not? If thats the case, there should be no separate changes entry.
          Hide
          Mark Miller added a comment -

          I've reopened SOLR-5099 as well - tests for these bugs are as important as the fixes.

          Show
          Mark Miller added a comment - I've reopened SOLR-5099 as well - tests for these bugs are as important as the fixes.
          Hide
          Erick Erickson added a comment -

          yeah, we should have a test, but this has been a pretty big rathole for me already and I didn't see a simple way to create a test, see my comment earlier.

          No, it didn't affect 4.4 so I'll take the entry out of CHANGES.txt in the next JIRA I fix (should be this morning sometime).

          Show
          Erick Erickson added a comment - yeah, we should have a test, but this has been a pretty big rathole for me already and I didn't see a simple way to create a test, see my comment earlier. No, it didn't affect 4.4 so I'll take the entry out of CHANGES.txt in the next JIRA I fix (should be this morning sometime).
          Hide
          Mark Miller added a comment -

          but this has been a pretty big rathole for me already and I didn't see a simple way to create a test

          That's fine, but please don't resolve the issue then. Bug fixes for really ugly issues like these absolutely need tests to make sure they don't keep coming back. We have seen that type of thing a lot recently - we fix something like this and it just breaks a couple months later in a new refactoring. You don't have to write the tests, but you might ask for some advice or help from someone else on it before resolving the issue. I'm happy to help make sure these problems have tests.

          Show
          Mark Miller added a comment - but this has been a pretty big rathole for me already and I didn't see a simple way to create a test That's fine, but please don't resolve the issue then. Bug fixes for really ugly issues like these absolutely need tests to make sure they don't keep coming back. We have seen that type of thing a lot recently - we fix something like this and it just breaks a couple months later in a new refactoring. You don't have to write the tests, but you might ask for some advice or help from someone else on it before resolving the issue. I'm happy to help make sure these problems have tests.
          Hide
          Erick Erickson added a comment -

          Well, the code is fixed, how about raising another JIRA instead?

          Show
          Erick Erickson added a comment - Well, the code is fixed, how about raising another JIRA instead?
          Hide
          Mark Miller added a comment -

          I don't consider this fixed without a test. The two issues are critical and somewhat complicated issues.

          I'm going to write the tests - without them, we only have your word they are fixed today and a random guess they will still be fixed tomorrow or the next day. These two issues are much too critical to not consider a test part of the issue.

          I'll finish the issues.

          Show
          Mark Miller added a comment - I don't consider this fixed without a test. The two issues are critical and somewhat complicated issues. I'm going to write the tests - without them, we only have your word they are fixed today and a random guess they will still be fixed tomorrow or the next day. These two issues are much too critical to not consider a test part of the issue. I'll finish the issues.
          Hide
          ASF subversion and git services added a comment -

          Commit 1514857 from Mark Miller in branch 'dev/trunk'
          [ https://svn.apache.org/r1514857 ]

          SOLR-5164: add relative solr.home testing to some tests, explicitly check for expected instanceDir handling with relative solr.home
          SOLR-5099: explicity check for proper solrcore.properties creation
          Speed up some tests by setting leaderVoteWait to 0

          Show
          ASF subversion and git services added a comment - Commit 1514857 from Mark Miller in branch 'dev/trunk' [ https://svn.apache.org/r1514857 ] SOLR-5164 : add relative solr.home testing to some tests, explicitly check for expected instanceDir handling with relative solr.home SOLR-5099 : explicity check for proper solrcore.properties creation Speed up some tests by setting leaderVoteWait to 0
          Hide
          ASF subversion and git services added a comment -

          Commit 1514858 from Mark Miller in branch 'dev/branches/branch_4x'
          [ https://svn.apache.org/r1514858 ]

          SOLR-5164: add relative solr.home testing to some tests, explicitly check for expected instanceDir handling with relative solr.home
          SOLR-5099: explicity check for proper solrcore.properties creation
          Speed up some tests by setting leaderVoteWait to 0

          Show
          ASF subversion and git services added a comment - Commit 1514858 from Mark Miller in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1514858 ] SOLR-5164 : add relative solr.home testing to some tests, explicitly check for expected instanceDir handling with relative solr.home SOLR-5099 : explicity check for proper solrcore.properties creation Speed up some tests by setting leaderVoteWait to 0
          Hide
          Mark Miller added a comment -

          I added some new important testing - we were not testing with a relative solr.home at all really - now it randomly uses one. I also added explicit testing to make sure the instance dir for collections api created cores is correct.

          Show
          Mark Miller added a comment - I added some new important testing - we were not testing with a relative solr.home at all really - now it randomly uses one. I also added explicit testing to make sure the instance dir for collections api created cores is correct.
          Hide
          ASF subversion and git services added a comment -

          Commit 1514939 from Mark Miller in branch 'dev/trunk'
          [ https://svn.apache.org/r1514939 ]

          SOLR-5164: fix relative solr.home testing on Windows

          Show
          ASF subversion and git services added a comment - Commit 1514939 from Mark Miller in branch 'dev/trunk' [ https://svn.apache.org/r1514939 ] SOLR-5164 : fix relative solr.home testing on Windows
          Hide
          ASF subversion and git services added a comment -

          Commit 1514940 from Mark Miller in branch 'dev/branches/branch_4x'
          [ https://svn.apache.org/r1514940 ]

          SOLR-5164: fix relative solr.home testing on Windows

          Show
          ASF subversion and git services added a comment - Commit 1514940 from Mark Miller in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1514940 ] SOLR-5164 : fix relative solr.home testing on Windows
          Hide
          Adrien Grand added a comment -

          4.5 release -> bulk close

          Show
          Adrien Grand added a comment - 4.5 release -> bulk close

            People

            • Assignee:
              Mark Miller
              Reporter:
              Erick Erickson
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development