Solr
  1. Solr
  2. SOLR-4478

Allow cores to specify a named config set in non-SolrCloud mode

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 4.2, 5.0
    • Fix Version/s: 4.9, 5.0
    • Component/s: None
    • Labels:
      None

      Description

      Part of moving forward to "the new way", after SOLR-4196 etc... I propose an additional parameter specified on the <core> node in solr.xml or as a parameter in the "discovery" mode core.properties file, call it configSet, where the value provided is a path to a directory, either absolute or relative. Really, this is as though you copied the conf directory somewhere to be used by more than one core.

      Straw-man: There will be a directory <solr_home>/configsets which will be the default. If the configSet parameter is, say, "myconf", then I'd expect a directory named "myconf" to exist in <solr_home>/configsets, which would look something like
      <solr_home>/configsets/myconf/schema.xml
      solrconfig.xml
      stopwords.txt
      velocity
      velocity/query.vm

      etc.

      If multiple cores used the same configSet, schema, solrconfig etc. would all be shared (i.e. shareSchema="true" would be assumed). I don't see a good use-case for not sharing schemas, so I don't propose to allow this to be turned off. Hmmm, what if shareSchema is explicitly set to false in the solr.xml or properties file? I'd guess it should be honored but maybe log a warning?

      Mostly I'm putting this up for comments. I know that there are already thoughts about how this all should work floating around, so before I start any work on this I thought I'd at least get an idea of whether this is the way people are thinking about going.

      Configset can be either a relative or absolute path, if relative it's assumed to be relative to <solr_home>.

      Thoughts?

      1. solr.log
        53 kB
        Shalin Shekhar Mangar
      2. SOLR-4478.patch
        29 kB
        Erick Erickson
      3. SOLR-4478.patch
        29 kB
        Erick Erickson
      4. SOLR-4478-take2.patch
        88 kB
        Alan Woodward
      5. SOLR-4478-take2.patch
        71 kB
        Alan Woodward
      6. SOLR-4478-take2.patch
        71 kB
        Alan Woodward
      7. SOLR-4478-take2.patch
        71 kB
        Alan Woodward

        Issue Links

          Activity

          Hide
          Steve Rowe added a comment -

          Erick, this is specifically for multicore, and not SolrCloud, right? I ask because this idea seems very much like the named configurations used per collection in SolrCloud.

          I have no idea if the two ideas could be unified, but if so, it might be worth exploring.

          Show
          Steve Rowe added a comment - Erick, this is specifically for multicore, and not SolrCloud, right? I ask because this idea seems very much like the named configurations used per collection in SolrCloud. I have no idea if the two ideas could be unified, but if so, it might be worth exploring.
          Hide
          Erick Erickson added a comment -

          Steve:

          Actually, it's the other way 'round. The idea of config sets for multcore had its origins in SolrCloud and this is being implemented (rather than some other options that were discussed several months ago) exactly to unify the two going forward.......

          Show
          Erick Erickson added a comment - Steve: Actually, it's the other way 'round. The idea of config sets for multcore had its origins in SolrCloud and this is being implemented (rather than some other options that were discussed several months ago) exactly to unify the two going forward.......
          Hide
          Yonik Seeley added a comment -

          The idea of config sets for multcore had its origins in SolrCloud

          Right, it would probably to make the most sense to name "configsets" as "configs" to match the layout in ZK.

          Show
          Yonik Seeley added a comment - The idea of config sets for multcore had its origins in SolrCloud Right, it would probably to make the most sense to name "configsets" as "configs" to match the layout in ZK.
          Hide
          Steve Rowe added a comment -

          Actually, it's the other way 'round. The idea of config sets for multcore had its origins in SolrCloud and this is being implemented (rather than some other options that were discussed several months ago) exactly to unify the two going forward.......

          Cool! Sorry for the Solr newb question

          Show
          Steve Rowe added a comment - Actually, it's the other way 'round. The idea of config sets for multcore had its origins in SolrCloud and this is being implemented (rather than some other options that were discussed several months ago) exactly to unify the two going forward....... Cool! Sorry for the Solr newb question
          Hide
          Erick Erickson added a comment -

          @Steve:
          NP, there was a lot of discussion around all this, I'm just glad someone's paying attention...

          @Yonik:
          Noted. Glad you're paying attention too <G>.. I'm about to start working on this, so nothing's set in stone. So:
          1> directory is <solr_home>/configs. How deep does this go? Say I have a configuration named erick. Is the tree <solr_home>/configs/erick/conf/schema.xmlandaltherest or <solr_home>/configs/erick/schema.xmlandaltherest?
          2> property in core.properties is configName?

          Show
          Erick Erickson added a comment - @Steve: NP, there was a lot of discussion around all this, I'm just glad someone's paying attention... @Yonik: Noted. Glad you're paying attention too <G>.. I'm about to start working on this, so nothing's set in stone. So: 1> directory is <solr_home>/configs. How deep does this go? Say I have a configuration named erick. Is the tree <solr_home>/configs/erick/conf/schema.xmlandaltherest or <solr_home>/configs/erick/schema.xmlandaltherest? 2> property in core.properties is configName?
          Hide
          Erick Erickson added a comment -

          Starting on this finally, couple of points for discussion:

          What do we do with each of these if we find a configSet entry in the core.properties file?
          > instanceDir - nothing to do here except we don't look here for configuration files
          > dataDir - again, nothing. The meaning remains unchanged.
          > config - check that it exists in the config set and blow up if we don't find it.
          > schema - treat as config.

          for config and schema, it hurts my head to think of resolving relative paths, absolute paths, the relationship to solr_home, the relationship of referenced files ( stopwords, etc). At least for the first cut I want to allow the config and schema files to be a different name, but that's it. And require that they live in the configSet directory. Unless all of this just automagically happens through the resource loader.

          The properties entry in the core.properties file (doesn't depend on configSet) - does it make sense to have it any more at all? I propose we deprecate it.

          Is there a convenient place in the SolrCloud code that I can rip off? I'll look but I don't want to re-invent the wheel if I miss it....

          Show
          Erick Erickson added a comment - Starting on this finally, couple of points for discussion: What do we do with each of these if we find a configSet entry in the core.properties file? > instanceDir - nothing to do here except we don't look here for configuration files > dataDir - again, nothing. The meaning remains unchanged. > config - check that it exists in the config set and blow up if we don't find it. > schema - treat as config. for config and schema, it hurts my head to think of resolving relative paths, absolute paths, the relationship to solr_home, the relationship of referenced files ( stopwords, etc). At least for the first cut I want to allow the config and schema files to be a different name, but that's it. And require that they live in the configSet directory. Unless all of this just automagically happens through the resource loader. The properties entry in the core.properties file (doesn't depend on configSet) - does it make sense to have it any more at all? I propose we deprecate it. Is there a convenient place in the SolrCloud code that I can rip off? I'll look but I don't want to re-invent the wheel if I miss it....
          Hide
          Yonik Seeley added a comment -

          2> property in core.properties is configName?

          Yes, we need to align this with solr cloud config as much as possible.
          Having configSet in some places and configName in other places isn't so hot

          Show
          Yonik Seeley added a comment - 2> property in core.properties is configName? Yes, we need to align this with solr cloud config as much as possible. Having configSet in some places and configName in other places isn't so hot
          Hide
          Erick Erickson added a comment -

          Preliminary patch if anyone's interested, nocommits and all. I think it's not too far from ready, although the core reload bit is kind of a hack...

          Show
          Erick Erickson added a comment - Preliminary patch if anyone's interested, nocommits and all. I think it's not too far from ready, although the core reload bit is kind of a hack...
          Hide
          Erick Erickson added a comment - - edited

          Updated patch with a problem. First I had the bright idea to interleave the configset-style and new-style core.properties files so we'd get some added testing done in OpenCloseCoreStressTest. Tests passed first time! Except for the stack traces, turns out I was eating an exception in the test that I shouldn't have been. Fortunately it seems to be a stack trace only thrown by the new code. NOTE: There's a nocommit in OpenCloseCoreStressTest that forces all cores to be configset only for easier debugging on this issue.

          But looking at the stack trace, there's an NPE at SearchHandler.180 or so, this line:
          ShardHandler shardHandler1 = shardHandlerFactory.getShardHandler();

          Of course the shardHandlerFactory is null here.

          So two things:

          1> Does it even make sense to share the SolrConfig object? I can imagine all sorts of threading issues here, but don't know the underlying code well enough to know whether to be terrified or not.

          2> Any clue why the shardHandlerFactory would be null? Near as I can tell, the SolrResourceLoader.inform method is where the problem starts, it sets the "live" member variable and later the NPE happens since the "live" member var aborts processing in the newInstance method.

          And if it's as simple as giving each core a new ResourceLoader, is there any point or is the work required at that point enough that sharing the solrconfig isn't worth the effort.

          Of course it may just be a sequencing issue, but I'm a little lost today, any wisdom gratefully received.

          Show
          Erick Erickson added a comment - - edited Updated patch with a problem. First I had the bright idea to interleave the configset-style and new-style core.properties files so we'd get some added testing done in OpenCloseCoreStressTest. Tests passed first time! Except for the stack traces, turns out I was eating an exception in the test that I shouldn't have been. Fortunately it seems to be a stack trace only thrown by the new code. NOTE: There's a nocommit in OpenCloseCoreStressTest that forces all cores to be configset only for easier debugging on this issue. But looking at the stack trace, there's an NPE at SearchHandler.180 or so, this line: ShardHandler shardHandler1 = shardHandlerFactory.getShardHandler(); Of course the shardHandlerFactory is null here. So two things: 1> Does it even make sense to share the SolrConfig object? I can imagine all sorts of threading issues here, but don't know the underlying code well enough to know whether to be terrified or not. 2> Any clue why the shardHandlerFactory would be null? Near as I can tell, the SolrResourceLoader.inform method is where the problem starts, it sets the "live" member variable and later the NPE happens since the "live" member var aborts processing in the newInstance method. And if it's as simple as giving each core a new ResourceLoader, is there any point or is the work required at that point enough that sharing the solrconfig isn't worth the effort. Of course it may just be a sequencing issue, but I'm a little lost today, any wisdom gratefully received.
          Hide
          Erick Erickson added a comment -

          If anyone has some spare cycles to work on this, please let me know. It's not that I'm stymied, I'm just having a really hard time finding the cycles to work on it. I can get this patch compiling again (after Alan and Mark and I did some reorganizing of all the persistence stuff, this is probably in kind of poor shape).

          Otherwise I'll get to it when I can, but I've been in that state for a couple of months now. I'm not looking to pass this off to someone completely, unless someone really wants it all for themselves, just looking for some extra energy.

          Show
          Erick Erickson added a comment - If anyone has some spare cycles to work on this, please let me know. It's not that I'm stymied, I'm just having a really hard time finding the cycles to work on it. I can get this patch compiling again (after Alan and Mark and I did some reorganizing of all the persistence stuff, this is probably in kind of poor shape). Otherwise I'll get to it when I can, but I've been in that state for a couple of months now. I'm not looking to pass this off to someone completely, unless someone really wants it all for themselves, just looking for some extra energy.
          Hide
          Alan Woodward added a comment -

          SolrConfig should be immutable, so I think that's safe to share. The thing that worries me slightly here is the interaction between shared configsets and mutable schemas. If two cores are sharing a config set with a managed schema, and someone makes changes to the schema under one of them, does the other core pick up the changes as well? Or does each core create it's own resourceloader, schema, etc from the shared config?

          I think we definitely want to combine this with the ZK loading code. There are already a bunch of places that have 'if (zk) doThis else doThat' spaghetti in them.

          Show
          Alan Woodward added a comment - SolrConfig should be immutable, so I think that's safe to share. The thing that worries me slightly here is the interaction between shared configsets and mutable schemas. If two cores are sharing a config set with a managed schema, and someone makes changes to the schema under one of them, does the other core pick up the changes as well? Or does each core create it's own resourceloader, schema, etc from the shared config? I think we definitely want to combine this with the ZK loading code. There are already a bunch of places that have 'if (zk) doThis else doThat' spaghetti in them.
          Hide
          Yonik Seeley added a comment -

          If two cores are sharing a config set with a managed schema, and someone makes changes to the schema under one of them, does the other core pick up the changes as well?

          I think it should... It's one logical schema, just as if you changed it on disk and reloaded/restarted the cores. One physical schema == one logical schema. It also already effectively works this way in Cloud mode.

          Show
          Yonik Seeley added a comment - If two cores are sharing a config set with a managed schema, and someone makes changes to the schema under one of them, does the other core pick up the changes as well? I think it should... It's one logical schema, just as if you changed it on disk and reloaded/restarted the cores. One physical schema == one logical schema. It also already effectively works this way in Cloud mode.
          Hide
          Alan Woodward added a comment -

          OK, how about this for an idea? Configsets are always shared, and so changes in one schema are reflected in all of them. But we add an option at core creation time to use the configset as a template, rather than sharing it, which copies the configset contents into the new core's config directory (whether on the filesystem or in Zk). Maybe add a command that will do this after core creation as well. So you can share all your config, but if you later decide that one core needs to diverge, then you split it off with it's own setup.

          Show
          Alan Woodward added a comment - OK, how about this for an idea? Configsets are always shared, and so changes in one schema are reflected in all of them. But we add an option at core creation time to use the configset as a template, rather than sharing it, which copies the configset contents into the new core's config directory (whether on the filesystem or in Zk). Maybe add a command that will do this after core creation as well. So you can share all your config, but if you later decide that one core needs to diverge, then you split it off with it's own setup.
          Hide
          Erick Erickson added a comment -

          bq: we add an option at core creation time to use the configset as a template, rather than sharing it, which copies the configset contents into the new core's config directory

          -1 as an initial reaction as stated. I really don't want to have a command that creates a mixed set of config sets and local-to-core configurations, if they want that control they can do it manually. And the +1 below keeps things more congruent with SolrCloud.

          +1 if we change it slightly. Use a template, but copy it to the config set directory with a new name and use that.

          BTW, the tentative directory structure is
          <solr_home>/configs/configset1
          <solr_home>/configs/configset2
          and so on, so copying from one to another should be straight-forward.

          In fact it's an open question (at least to me) whether we support local-to-core configurations in 5.0 or require config sets. We could support both, which is used is controlled by the presence/absence of a "configName" parameter in the core definition (<core now, but configName in core.properties)

          Show
          Erick Erickson added a comment - bq: we add an option at core creation time to use the configset as a template, rather than sharing it, which copies the configset contents into the new core's config directory -1 as an initial reaction as stated. I really don't want to have a command that creates a mixed set of config sets and local-to-core configurations, if they want that control they can do it manually. And the +1 below keeps things more congruent with SolrCloud. +1 if we change it slightly. Use a template, but copy it to the config set directory with a new name and use that . BTW, the tentative directory structure is <solr_home>/configs/configset1 <solr_home>/configs/configset2 and so on, so copying from one to another should be straight-forward. In fact it's an open question (at least to me) whether we support local-to-core configurations in 5.0 or require config sets. We could support both, which is used is controlled by the presence/absence of a "configName" parameter in the core definition (<core now, but configName in core.properties)
          Hide
          Steve Rowe added a comment -

          If two cores are sharing a config set with a managed schema, and someone makes changes to the schema under one of them, does the other core pick up the changes as well?

          I think it should... It's one logical schema, just as if you changed it on disk and reloaded/restarted the cores. One physical schema == one logical schema. It also already effectively works this way in Cloud mode.

          +1

          In my original commit for SOLR-3251, I had code to handle shared + mutable schemas, but it wasn't all hooked up properly or tested, and with named config sets in play, I haven't pursued it.

          For local shared + mutable schema, I think we have two choices:

          1. Each named config set has a single shared in-memory representation, in addition to a single shared persisted representation.
          2. Each core has its own private in-memory representation, updated when the shared persisted representation is updated.

          #2 nullifies the utility of shared schemas, so it's not really an option, IMHO.

          Show
          Steve Rowe added a comment - If two cores are sharing a config set with a managed schema, and someone makes changes to the schema under one of them, does the other core pick up the changes as well? I think it should... It's one logical schema, just as if you changed it on disk and reloaded/restarted the cores. One physical schema == one logical schema. It also already effectively works this way in Cloud mode. +1 In my original commit for SOLR-3251 , I had code to handle shared + mutable schemas, but it wasn't all hooked up properly or tested, and with named config sets in play, I haven't pursued it. For local shared + mutable schema, I think we have two choices: Each named config set has a single shared in-memory representation, in addition to a single shared persisted representation. Each core has its own private in-memory representation, updated when the shared persisted representation is updated. #2 nullifies the utility of shared schemas, so it's not really an option, IMHO.
          Hide
          Alan Woodward added a comment -

          Have found another problem here - what to do with core-specific properties? Core properties are passed to the SolrConfig object at construction, so there's no way at present to use a new set of properties with an existing configset. Same with IndexSchema, which re-uses the resource loader from SolrConfig.

          Show
          Alan Woodward added a comment - Have found another problem here - what to do with core-specific properties? Core properties are passed to the SolrConfig object at construction, so there's no way at present to use a new set of properties with an existing configset. Same with IndexSchema, which re-uses the resource loader from SolrConfig.
          Hide
          Erick Erickson added a comment -

          OK, reconstructing an chat exchange:

          Sharing the underlying solrconfig objects looks like it's more difficult than I thought, with some interesting corner cases that would be difficult, i.e ${} substitutions, resource loader being shared, etc. Also, the individual core properties are embedded in the Config object, so keeping these separate is another source of getting code wrong.

          Not to mention that the code changes would be more extensive than anyone had hoped.

          At lest the use-case of opening a core and actively using it for a while then moving on is handled by the lazy/transient core opportunities.

          There is historical evidence that a significant amount of CPU resources are consumed by opening/closing cores 100s of times a second, so that scenario is still out there.

          The net-net is that it's probably not worth the effort right now to really share the underlying solrConfig object across cores, too many ways to go wrong. The refactoring that's been done should make this easier if we decide to do it in the future.

          Show
          Erick Erickson added a comment - OK, reconstructing an chat exchange: Sharing the underlying solrconfig objects looks like it's more difficult than I thought, with some interesting corner cases that would be difficult, i.e ${} substitutions, resource loader being shared, etc. Also, the individual core properties are embedded in the Config object, so keeping these separate is another source of getting code wrong. Not to mention that the code changes would be more extensive than anyone had hoped. At lest the use-case of opening a core and actively using it for a while then moving on is handled by the lazy/transient core opportunities. There is historical evidence that a significant amount of CPU resources are consumed by opening/closing cores 100s of times a second, so that scenario is still out there. The net-net is that it's probably not worth the effort right now to really share the underlying solrConfig object across cores, too many ways to go wrong. The refactoring that's been done should make this easier if we decide to do it in the future.
          Hide
          Erick Erickson added a comment -

          I just had a bright idea, so I'll put it out there so someone can shoot it down. It seems like sharing the underlying solrConfig object is fraught with problems, but could we get an easy win by just sharing the parsed DOM object in each config set (really, same for schema object?) I don't have any measurements for what percentage of loading the schema object is spent in raw XML parsing, so I can't really say how much of a win this would be. But if it's easy/safe it might be worth considering.

          Show
          Erick Erickson added a comment - I just had a bright idea, so I'll put it out there so someone can shoot it down. It seems like sharing the underlying solrConfig object is fraught with problems, but could we get an easy win by just sharing the parsed DOM object in each config set (really, same for schema object?) I don't have any measurements for what percentage of loading the schema object is spent in raw XML parsing, so I can't really say how much of a win this would be. But if it's easy/safe it might be worth considering.
          Hide
          Noble Paul added a comment -

          handling the ${} substitutions are more work. But a lot of users who need sharing of solrconfig would be happy to sacrifice that feature. if there are ${} is solrconfig.xml and shareConfig=true we can just fail.

          Longer term we can support that too but will need some more changes the way properties are read

          Show
          Noble Paul added a comment - handling the ${} substitutions are more work. But a lot of users who need sharing of solrconfig would be happy to sacrifice that feature. if there are ${} is solrconfig.xml and shareConfig=true we can just fail. Longer term we can support that too but will need some more changes the way properties are read
          Hide
          Erick Erickson added a comment -

          I got to thinking about this and trying to take it out of mothballs and I'm starting to think it's a terrible idea for 4.x and should be postponed or abandoned unless and until we do something like what has been discussed elsewhere; having there be "one source of truth" (ZooKeeper has been discussed for instance). So I'll list out the issues I've thought about and if there are straightforward answers to them I'll be happy to reconsider.

          Each issue is probably technically do-able, but the sum (and ones I haven't seen yet) totally scare me.

          1> Traditional master/slave architectures. Let's say we change the schema (it'd have to be on the master?). How to get that to the slaves? Currently the confFiles directive has an explicit test and will not copy a directory. I'm not convinced it'd even work with relative paths and listing every file in the configset dir would be kludgy at best. And I think the confFiles directive doesn't work outside the "conf" directory for the core it's replicating anyway. I suppose the user could copy the configset directory to all the nodes in the farm, but....

          2> The new REST API for modifying the schema. In non-SolrCloud mode, how does that work? Is it only allowed on the master (assuming we can solve <1>)? How to enforce?

          3> Sharing the solrConfig object is also fraught with issues as discussed above. There's already the "share schema" option, so at least it's possible to have one shared schema.

          4> How to get any changes reloaded in a master/slave environment for all the affected cores on all the machines? You'd need some kind of manual process of going to each one and issuing a new command "ReloadAllCores" or build in some kind of notification system. Or we'd need to require the user to keep a list of all the nodes and all the cores and script reloading them all. Nobody should be re-inventing ZooKeeper.

          5> How to get any changes reloaded in even the non master/slave environment for all the affected cores? A new command? Periodic polling? Check every query/update request?

          6> Sticky wickets I haven't thought of yet, I'm afraid, very afraid... Each of these is solvable, but considering the effort involved it doesn't seem like it's worth pursuing right now, at least my interest is disappearing.

          And wrapped around this is that SolrCloud already handles most of the things I'm worried about, especially getting changes propagated to all the right places in the cluster. SolrCloud already has a way to reload all the nodes that take part in a collection. SolrCloud already has the notifications of changes to the config set built in (at least I think, if not it will).

          My feeling at this point is that supporting this well would turn into a huge amount of work that would then be thrown away if we go to a "one source of truth" model in Solr5 (or even 6). And that actually using the capability would be fragile and complex. So unless I can be convinced otherwise, I'm going to assign this back to nobody and forget about it.

          Show
          Erick Erickson added a comment - I got to thinking about this and trying to take it out of mothballs and I'm starting to think it's a terrible idea for 4.x and should be postponed or abandoned unless and until we do something like what has been discussed elsewhere; having there be "one source of truth" (ZooKeeper has been discussed for instance). So I'll list out the issues I've thought about and if there are straightforward answers to them I'll be happy to reconsider. Each issue is probably technically do-able, but the sum (and ones I haven't seen yet) totally scare me. 1> Traditional master/slave architectures. Let's say we change the schema (it'd have to be on the master?). How to get that to the slaves? Currently the confFiles directive has an explicit test and will not copy a directory. I'm not convinced it'd even work with relative paths and listing every file in the configset dir would be kludgy at best. And I think the confFiles directive doesn't work outside the "conf" directory for the core it's replicating anyway. I suppose the user could copy the configset directory to all the nodes in the farm, but.... 2> The new REST API for modifying the schema. In non-SolrCloud mode, how does that work? Is it only allowed on the master (assuming we can solve <1>)? How to enforce? 3> Sharing the solrConfig object is also fraught with issues as discussed above. There's already the "share schema" option, so at least it's possible to have one shared schema. 4> How to get any changes reloaded in a master/slave environment for all the affected cores on all the machines? You'd need some kind of manual process of going to each one and issuing a new command "ReloadAllCores" or build in some kind of notification system. Or we'd need to require the user to keep a list of all the nodes and all the cores and script reloading them all. Nobody should be re-inventing ZooKeeper. 5> How to get any changes reloaded in even the non master/slave environment for all the affected cores? A new command? Periodic polling? Check every query/update request? 6> Sticky wickets I haven't thought of yet, I'm afraid, very afraid... Each of these is solvable, but considering the effort involved it doesn't seem like it's worth pursuing right now, at least my interest is disappearing. And wrapped around this is that SolrCloud already handles most of the things I'm worried about, especially getting changes propagated to all the right places in the cluster. SolrCloud already has a way to reload all the nodes that take part in a collection. SolrCloud already has the notifications of changes to the config set built in (at least I think, if not it will). My feeling at this point is that supporting this well would turn into a huge amount of work that would then be thrown away if we go to a "one source of truth" model in Solr5 (or even 6). And that actually using the capability would be fragile and complex. So unless I can be convinced otherwise, I'm going to assign this back to nobody and forget about it.
          Hide
          Trey Grainger added a comment -

          (moving this from my previous e-mail to the solr-dev mailing list)

          There are two use-cases that appear broken with the new core auto-discovery mechanism:

          1) The Core Admin Handler's CREATE command no longer works to create brand new cores
          (unless you have logged on the box and created the core's directory structure manually, which largely defeats the purpose of the "CREATE" command). With the old Solr.xml format, we could spin up as many cores as we wanted to dynamically with the following command:
          http://localhost:8983/solr/admin/cores?action=CREATE&name=newCore1&instanceDir=collection1&dataDir=newCore1/data
          ...
          http://localhost:8983/solr/admin/cores?action=CREATE&name=newCoreN&instanceDir=collection1&dataDir=newCoreN/data

          In the new core discovery mode, this exception is now thrown:
          Error CREATEing SolrCore 'newCore1': Could not create a new core in solr/collection1/as another core is already defined there

          The exception is being intentionally thrown in CorePropertiesLocator.java because a core.properties file already exists in solr/collection1 (and only one can exist per directory).

          2) Having a shared configuration directory (instanceDir) across many cores no longer works.
          Every core has to have it's own conf/ directory, and this doesn't seem to be overridable any longer. Previously, it was possible to have many cores share the same instanceDir (and just override their dataDir for obvious reasons). Now, it is necessary to copy and paste identical config files for each Solr core.

          I don't know if there's already a current roadmap for fixing this. I saw https://issues.apache.org/jira/browse/SOLR-4478, which suggested replacing instanceDir with the ability to specify a named configSet. This solves problem 2, but not problem1 (since you still can't have multiple core.properties files in the same folder). Based on Erick's comments in the JIRA ticket, it also sounds like this ticket is also dead at the moment.

          There is definitely a need to have a shared config directory - whether that is through a configSet or an explicit indexDir doesn't matter to me. There's also a need to be able to dynamically create Solr cores from external systems. I currently can't upgrade to core auto discovery because it doesn't allow dynamic core creation. Does anyone have some thoughts on how to best get these features working again under core autodiscovery? Adding instanceDir to core.properties seems like an easy solution, but there must be a desire not to do that or it would probably have already been done.

          I'm happy to contribute some time to resolving this if there is agreed upon path forward.

          Show
          Trey Grainger added a comment - (moving this from my previous e-mail to the solr-dev mailing list) There are two use-cases that appear broken with the new core auto-discovery mechanism: 1) The Core Admin Handler's CREATE command no longer works to create brand new cores (unless you have logged on the box and created the core's directory structure manually, which largely defeats the purpose of the "CREATE" command). With the old Solr.xml format, we could spin up as many cores as we wanted to dynamically with the following command: http://localhost:8983/solr/admin/cores?action=CREATE&name=newCore1&instanceDir=collection1&dataDir=newCore1/data ... http://localhost:8983/solr/admin/cores?action=CREATE&name=newCoreN&instanceDir=collection1&dataDir=newCoreN/data In the new core discovery mode, this exception is now thrown: Error CREATEing SolrCore 'newCore1': Could not create a new core in solr/collection1/as another core is already defined there The exception is being intentionally thrown in CorePropertiesLocator.java because a core.properties file already exists in solr/collection1 (and only one can exist per directory). 2) Having a shared configuration directory (instanceDir) across many cores no longer works. Every core has to have it's own conf/ directory, and this doesn't seem to be overridable any longer. Previously, it was possible to have many cores share the same instanceDir (and just override their dataDir for obvious reasons). Now, it is necessary to copy and paste identical config files for each Solr core. I don't know if there's already a current roadmap for fixing this. I saw https://issues.apache.org/jira/browse/SOLR-4478 , which suggested replacing instanceDir with the ability to specify a named configSet. This solves problem 2, but not problem1 (since you still can't have multiple core.properties files in the same folder). Based on Erick's comments in the JIRA ticket, it also sounds like this ticket is also dead at the moment. There is definitely a need to have a shared config directory - whether that is through a configSet or an explicit indexDir doesn't matter to me. There's also a need to be able to dynamically create Solr cores from external systems. I currently can't upgrade to core auto discovery because it doesn't allow dynamic core creation. Does anyone have some thoughts on how to best get these features working again under core autodiscovery? Adding instanceDir to core.properties seems like an easy solution, but there must be a desire not to do that or it would probably have already been done. I'm happy to contribute some time to resolving this if there is agreed upon path forward.
          Hide
          Trey Grainger added a comment - - edited

          (Erick's response to my post)

          Right, let's move this discussion to SOLR-4779. There's some history
          here. Sharing named config sets got a bit wrapped up in sharing the
          underlying solrconfig object. This latter has been taken off the
          table, but we should discuss fixing Trey's issues up. Here's what the
          thinking was:
          There would be a directory like <solr_home>/configs/configset1,
          <solr_home>/configs/configset2, etc. Then a new parameter for
          core.properties or create or whatever like "configset=configset1" that
          would be smart enough to look in <solr_home>/configs for an entire
          conf directory named "configste1".

          Trey:
          Does that work for your case? If so, please add your comments to 4779
          and we can take it from there. FWIW, I don't think this is especially
          hard, but time is always at a premium.

          Show
          Trey Grainger added a comment - - edited (Erick's response to my post) Right, let's move this discussion to SOLR-4779 . There's some history here. Sharing named config sets got a bit wrapped up in sharing the underlying solrconfig object. This latter has been taken off the table, but we should discuss fixing Trey's issues up. Here's what the thinking was: There would be a directory like <solr_home>/configs/configset1, <solr_home>/configs/configset2, etc. Then a new parameter for core.properties or create or whatever like "configset=configset1" that would be smart enough to look in <solr_home>/configs for an entire conf directory named "configste1". Trey: Does that work for your case? If so, please add your comments to 4779 and we can take it from there. FWIW, I don't think this is especially hard, but time is always at a premium.
          Hide
          Trey Grainger added a comment - - edited

          Hi Erick,

          Yes, that resolves the hardest of the two problems. The other issue is that since a dedicated folder is now required per-core (to hold the core.properties file), the core CREATE command needs to now also be able to create the folder for the new core if it doesn't exist. Something like:
          http://localhost:8983/solr/admin/cores?action=CREATE&name=newCore& coreDir=cores/newCore &configset=sharedconfig

          Alternatively, instanceDir could continue to serve that function (instead of being deprecated):
          http://localhost:8983/solr/admin/cores?action=CREATE&name=newCore& instanceDir=cores/newCore &configset=sharedconfig

          I think the combination of adding configSet and adding the ability for the CREATE command to actually create the new folder to hold core.properties should handle the use case.

          Show
          Trey Grainger added a comment - - edited Hi Erick, Yes, that resolves the hardest of the two problems. The other issue is that since a dedicated folder is now required per-core (to hold the core.properties file), the core CREATE command needs to now also be able to create the folder for the new core if it doesn't exist. Something like: http://localhost:8983/solr/admin/cores?action=CREATE&name=newCore& coreDir=cores/newCore &configset=sharedconfig Alternatively, instanceDir could continue to serve that function (instead of being deprecated): http://localhost:8983/solr/admin/cores?action=CREATE&name=newCore& instanceDir=cores/newCore &configset=sharedconfig I think the combination of adding configSet and adding the ability for the CREATE command to actually create the new folder to hold core.properties should handle the use case.
          Hide
          Alan Woodward added a comment -

          I got some spare cycles, and had another stab at this.

          • new ConfigSet object that contains a SolrConfig and IndexSchema
          • ConfigSet loading and discovery is dealt with by a ConfigSetService, which comes in Cloud, Default and SchemaCaching varieties.
          • Config sets are kept in solrhome/configsets by default, but this can be configured in solr.xml
          • The actual schema and config objects are not shared between cores, unless the share schema flag is switched on.
          • Concurrency in schema sharing is dealt with by using a loading cache from Guava.

          This ends up tidying up some of the zookeeper/not zookeeper logic in CoreContainer as well, which is nice. Tests are passing so far...

          Show
          Alan Woodward added a comment - I got some spare cycles, and had another stab at this. new ConfigSet object that contains a SolrConfig and IndexSchema ConfigSet loading and discovery is dealt with by a ConfigSetService, which comes in Cloud, Default and SchemaCaching varieties. Config sets are kept in solrhome/configsets by default, but this can be configured in solr.xml The actual schema and config objects are not shared between cores, unless the share schema flag is switched on. Concurrency in schema sharing is dealt with by using a loading cache from Guava. This ends up tidying up some of the zookeeper/not zookeeper logic in CoreContainer as well, which is nice. Tests are passing so far...
          Hide
          Alan Woodward added a comment -

          @erickerickson do you have any comments on the latest patch? I'd like to add some initial tests for the SnapPuller, but because this does everything via the SolrResourceLoader I think replication should just work.

          This should also make writing tests a lot easier, as you can just point to a configset rather than having to copy files around when spinning up test cores.

          Show
          Alan Woodward added a comment - @erickerickson do you have any comments on the latest patch? I'd like to add some initial tests for the SnapPuller, but because this does everything via the SolrResourceLoader I think replication should just work. This should also make writing tests a lot easier, as you can just point to a configset rather than having to copy files around when spinning up test cores.
          Hide
          Erick Erickson added a comment -

          Oh, man! I spent all of December sailing from the Canary Islands to Antigua. And now we're selling our house (closing 17-Jan) and driving to California. So I haven't looked at much of anything lately.....

          Glad you prompted me. We'll be driving out to CA over the next couple of weeks, so I should be able to spend some time looking at this, but not before Sunday.....

          Show
          Erick Erickson added a comment - Oh, man! I spent all of December sailing from the Canary Islands to Antigua. And now we're selling our house (closing 17-Jan) and driving to California. So I haven't looked at much of anything lately..... Glad you prompted me. We'll be driving out to CA over the next couple of weeks, so I should be able to spend some time looking at this, but not before Sunday.....
          Hide
          Erick Erickson added a comment -

          Alan has some cycles for this and wants to work on it....

          Show
          Erick Erickson added a comment - Alan has some cycles for this and wants to work on it....
          Hide
          Alan Woodward added a comment -

          Patch updated to trunk.

          SnapPuller is still giving me pause here. It allows you to define a list of config files that you want to replicate, and is hardcoded to store them under the core instancedir. Maybe the thing to do is to add a writeConfigFile() method to SolrResourceLoader, and throw UOE on the ZkSolrResourceLoader implementation? That way the replicated file is always written to the write place (instancedir or configset).

          Show
          Alan Woodward added a comment - Patch updated to trunk. SnapPuller is still giving me pause here. It allows you to define a list of config files that you want to replicate, and is hardcoded to store them under the core instancedir. Maybe the thing to do is to add a writeConfigFile() method to SolrResourceLoader, and throw UOE on the ZkSolrResourceLoader implementation? That way the replicated file is always written to the write place (instancedir or configset).
          Hide
          Alan Woodward added a comment -

          I tell a lie, the SnapPuller actually writes to the dir specified by SolrCore.getResourceLoader().getConfigDir(), so it should work properly here. Will try and write a test that exercises it properly though.

          Show
          Alan Woodward added a comment - I tell a lie, the SnapPuller actually writes to the dir specified by SolrCore.getResourceLoader().getConfigDir(), so it should work properly here. Will try and write a test that exercises it properly though.
          Hide
          Alan Woodward added a comment -

          Patch updated to trunk.

          The replicationhandler tests don't actually test anything in separate corecontainers at the moment, and it looks like it will be a bit of work to get that done, so I'd like to open a separate JIRA for that, and commit this asap. I've tested this locally with a standalone setup and replication deals with this without any problems that I've found.

          I'll commit this patch tomorrow if there are no objections.

          Show
          Alan Woodward added a comment - Patch updated to trunk. The replicationhandler tests don't actually test anything in separate corecontainers at the moment, and it looks like it will be a bit of work to get that done, so I'd like to open a separate JIRA for that, and commit this asap. I've tested this locally with a standalone setup and replication deals with this without any problems that I've found. I'll commit this patch tomorrow if there are no objections.
          Hide
          Alan Woodward added a comment -

          New patch, adding CoreAdminHandler and solrj integration.

          Show
          Alan Woodward added a comment - New patch, adding CoreAdminHandler and solrj integration.
          Hide
          ASF subversion and git services added a comment -

          Commit 1580814 from Alan Woodward in branch 'dev/trunk'
          [ https://svn.apache.org/r1580814 ]

          SOLR-4478: Allow cores to use configurations specified outside their instance directory

          Show
          ASF subversion and git services added a comment - Commit 1580814 from Alan Woodward in branch 'dev/trunk' [ https://svn.apache.org/r1580814 ] SOLR-4478 : Allow cores to use configurations specified outside their instance directory
          Hide
          ASF subversion and git services added a comment -

          Commit 1580817 from Alan Woodward in branch 'dev/branches/branch_4x'
          [ https://svn.apache.org/r1580817 ]

          SOLR-4478: Allow cores to use configurations specified outside their instance directory

          Show
          ASF subversion and git services added a comment - Commit 1580817 from Alan Woodward in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1580817 ] SOLR-4478 : Allow cores to use configurations specified outside their instance directory
          Hide
          Alan Woodward added a comment -

          OK, so "tomorrow" turned into "next week", but there we go. I'll hold the JIRA open until I've written something for the reference guide.

          Show
          Alan Woodward added a comment - OK, so "tomorrow" turned into "next week", but there we go. I'll hold the JIRA open until I've written something for the reference guide.
          Hide
          Erick Erickson added a comment -

          Thanks! I think this'll be cool.

          I can hardly complain, I didn't get around to this for a year ..

          Probably should have an entry in the new CWiki form of the documentation too? I'm finding it extremely helpful to be able to download all that in PDF, have a single place to search the current docs (rather than Google and get something from Solr 1.4) etc.... many kudos to Cassandra & Co.

          Show
          Erick Erickson added a comment - Thanks! I think this'll be cool. I can hardly complain, I didn't get around to this for a year .. Probably should have an entry in the new CWiki form of the documentation too? I'm finding it extremely helpful to be able to download all that in PDF, have a single place to search the current docs (rather than Google and get something from Solr 1.4) etc.... many kudos to Cassandra & Co.
          Hide
          ASF subversion and git services added a comment -

          Commit 1580839 from Alan Woodward in branch 'dev/trunk'
          [ https://svn.apache.org/r1580839 ]

          SOLR-4478: Test fix for Windows filepaths

          Show
          ASF subversion and git services added a comment - Commit 1580839 from Alan Woodward in branch 'dev/trunk' [ https://svn.apache.org/r1580839 ] SOLR-4478 : Test fix for Windows filepaths
          Hide
          ASF subversion and git services added a comment -

          Commit 1580841 from Alan Woodward in branch 'dev/branches/branch_4x'
          [ https://svn.apache.org/r1580841 ]

          SOLR-4478: Test fix for Windows filepaths

          Show
          ASF subversion and git services added a comment - Commit 1580841 from Alan Woodward in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1580841 ] SOLR-4478 : Test fix for Windows filepaths
          Hide
          Alan Woodward added a comment -

          There's a few follow up things I'd like to do as well, like managing configsets via a RestManager.

          I'm not sure if I've got the right permissions to edit the CWiki yet though? (have just created a login under 'romseygeek').

          Show
          Alan Woodward added a comment - There's a few follow up things I'd like to do as well, like managing configsets via a RestManager. I'm not sure if I've got the right permissions to edit the CWiki yet though? (have just created a login under 'romseygeek').
          Hide
          Cassandra Targett added a comment -

          Alan Woodward

          I'm not sure if I've got the right permissions to edit the CWiki yet though?

          You're a committer, so you can get the permissions. One of the PMC guys need to give them to you though, so email your Confluence account name to the PMC (private@lucene) requesting Confluence edit permissions.

          Show
          Cassandra Targett added a comment - Alan Woodward I'm not sure if I've got the right permissions to edit the CWiki yet though? You're a committer, so you can get the permissions. One of the PMC guys need to give them to you though, so email your Confluence account name to the PMC (private@lucene) requesting Confluence edit permissions.
          Hide
          Alan Woodward added a comment -

          Thanks Cassandra, have emailed.

          Show
          Alan Woodward added a comment - Thanks Cassandra, have emailed.
          Hide
          Shalin Shekhar Mangar added a comment -

          I'm not sure but I think this is causing problems in SolrCloud mode. In both trunk and branch_4x, a simple SolrCloud cluster fails to start with:

          1. cd solr
          2. ant example
          3. java -Dbootstrap_confdir=./solr/collection1/conf -Dcollection.configName=myconf -DzkRun -DnumShards=1 -jar start.jar

            4392 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore – QuerySenderListener sending requests to Searcher@1ad91a94[collection1] main

            Unknown macro: {StandardDirectoryReader(segments_1}

            4395 [searcherExecutor-5-thread-1] ERROR org.apache.solr.core.SolrCore – java.lang.NullPointerException
            at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:194)
            at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
            at org.apache.solr.core.SolrCore.execute(SolrCore.java:1952)
            at org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:64)
            at org.apache.solr.core.SolrCore$5.call(SolrCore.java:1724)
            at java.util.concurrent.FutureTask.run(FutureTask.java:262)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
            at java.lang.Thread.run(Thread.java:744)

          The lucene_solr_4_7 branch does not have this problem.

          Show
          Shalin Shekhar Mangar added a comment - I'm not sure but I think this is causing problems in SolrCloud mode. In both trunk and branch_4x, a simple SolrCloud cluster fails to start with: cd solr ant example java -Dbootstrap_confdir=./solr/collection1/conf -Dcollection.configName=myconf -DzkRun -DnumShards=1 -jar start.jar 4392 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore – QuerySenderListener sending requests to Searcher@1ad91a94 [collection1] main Unknown macro: {StandardDirectoryReader(segments_1} 4395 [searcherExecutor-5-thread-1] ERROR org.apache.solr.core.SolrCore – java.lang.NullPointerException at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:194) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1952) at org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:64) at org.apache.solr.core.SolrCore$5.call(SolrCore.java:1724) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) The lucene_solr_4_7 branch does not have this problem.
          Hide
          Alan Woodward added a comment -

          The shardhandlerfactory isn't getting set, which implies that inform() isn't being called for some reason. I'll dig...

          Show
          Alan Woodward added a comment - The shardhandlerfactory isn't getting set, which implies that inform() isn't being called for some reason. I'll dig...
          Hide
          Alan Woodward added a comment -

          I can't reproduce this on trunk - are you doing this on a fresh checkout Shalin?

          Show
          Alan Woodward added a comment - I can't reproduce this on trunk - are you doing this on a fresh checkout Shalin?
          Hide
          Shalin Shekhar Mangar added a comment -

          I can't reproduce this on trunk - are you doing this on a fresh checkout Shalin?

          Yes, I did an svn update on both branch_4x and trunk again. ant check-svn-working-copy was successful. I can reproduce it on two different machines (mac and linux) on both trunk and branch_4x. I think it is timing specific because out of four tries, one started up successfully and three failed.

          Show
          Shalin Shekhar Mangar added a comment - I can't reproduce this on trunk - are you doing this on a fresh checkout Shalin? Yes, I did an svn update on both branch_4x and trunk again. ant check-svn-working-copy was successful. I can reproduce it on two different machines (mac and linux) on both trunk and branch_4x. I think it is timing specific because out of four tries, one started up successfully and three failed.
          Hide
          Shalin Shekhar Mangar added a comment -

          I also have this in the logs:

          4243 [coreLoadExecutor-4-thread-1] ERROR org.apache.solr.core.CoreContainer – null:org.apache.solr.common.SolrException: Unable to create core: collection1
          at org.apache.solr.core.CoreContainer.recordAndThrow(CoreContainer.java:911)
          at org.apache.solr.core.CoreContainer.create(CoreContainer.java:568)
          at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:261)
          at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:253)
          at java.util.concurrent.FutureTask.run(FutureTask.java:262)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
          at java.util.concurrent.FutureTask.run(FutureTask.java:262)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
          at java.lang.Thread.run(Thread.java:744)
          Caused by: org.apache.solr.common.SolrException: Error initializing QueryElevationComponent.
          at org.apache.solr.core.SolrCore.<init>(SolrCore.java:845)
          at org.apache.solr.core.SolrCore.<init>(SolrCore.java:631)
          at org.apache.solr.core.CoreContainer.create(CoreContainer.java:556)
          ... 8 more
          Caused by: org.apache.solr.common.SolrException: Error initializing QueryElevationComponent.
          at org.apache.solr.handler.component.QueryElevationComponent.inform(QueryElevationComponent.java:243)
          at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:631)
          at org.apache.solr.core.SolrCore.<init>(SolrCore.java:836)
          ... 10 more
          Caused by: org.apache.solr.common.SolrException: Error loading config name for collection collection1
          at org.apache.solr.common.cloud.ZkStateReader.readConfigName(ZkStateReader.java:167)
          at org.apache.solr.handler.component.QueryElevationComponent.inform(QueryElevationComponent.java:213)
          ... 12 more
          Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1
          at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
          at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
          at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
          at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:276)
          at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:273)
          at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
          at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:273)
          at org.apache.solr.common.cloud.ZkStateReader.readConfigName(ZkStateReader.java:147)
          ... 13 more

          Show
          Shalin Shekhar Mangar added a comment - I also have this in the logs: 4243 [coreLoadExecutor-4-thread-1] ERROR org.apache.solr.core.CoreContainer – null:org.apache.solr.common.SolrException: Unable to create core: collection1 at org.apache.solr.core.CoreContainer.recordAndThrow(CoreContainer.java:911) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:568) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:261) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:253) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Caused by: org.apache.solr.common.SolrException: Error initializing QueryElevationComponent. at org.apache.solr.core.SolrCore.<init>(SolrCore.java:845) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:631) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:556) ... 8 more Caused by: org.apache.solr.common.SolrException: Error initializing QueryElevationComponent. at org.apache.solr.handler.component.QueryElevationComponent.inform(QueryElevationComponent.java:243) at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:631) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:836) ... 10 more Caused by: org.apache.solr.common.SolrException: Error loading config name for collection collection1 at org.apache.solr.common.cloud.ZkStateReader.readConfigName(ZkStateReader.java:167) at org.apache.solr.handler.component.QueryElevationComponent.inform(QueryElevationComponent.java:213) ... 12 more Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1 at org.apache.zookeeper.KeeperException.create(KeeperException.java:111) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155) at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:276) at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:273) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73) at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:273) at org.apache.solr.common.cloud.ZkStateReader.readConfigName(ZkStateReader.java:147) ... 13 more
          Hide
          Alan Woodward added a comment - - edited

          Huh, I can't reproduce it all on MacOS. Could you attach the whole log output?

          Show
          Alan Woodward added a comment - - edited Huh, I can't reproduce it all on MacOS. Could you attach the whole log output?
          Hide
          Shalin Shekhar Mangar added a comment -

          Attaching solr.log from trunk on mac. Run command was:
          java -Dbootstrap_confdir=./solr/collection1/conf -Dcollection.configName=myconf -DzkRun -DnumShards=1 -jar start.jar

          Show
          Shalin Shekhar Mangar added a comment - Attaching solr.log from trunk on mac. Run command was: java -Dbootstrap_confdir=./solr/collection1/conf -Dcollection.configName=myconf -DzkRun -DnumShards=1 -jar start.jar
          Hide
          Alan Woodward added a comment -

          Aha, now I can reproduce. I wasn't running from a clean checkout, and had an old zk data directory hanging about. Apologies.

          So the root error is the core starting up and trying to find a collection called collection1 to attach the core to, but not finding it because it hasn't been created yet. I'm slightly confused as to why it is working in 4.7 though. Will keep looking.

          Show
          Alan Woodward added a comment - Aha, now I can reproduce. I wasn't running from a clean checkout, and had an old zk data directory hanging about. Apologies. So the root error is the core starting up and trying to find a collection called collection1 to attach the core to, but not finding it because it hasn't been created yet. I'm slightly confused as to why it is working in 4.7 though. Will keep looking.
          Hide
          Alan Woodward added a comment -

          OK, so the previous behaviour was to automatically create the collection in ZK if it doesn't exist for a given core. I'd missed transferring that part to the CloudConfigSetService. Working up a patch now.

          Show
          Alan Woodward added a comment - OK, so the previous behaviour was to automatically create the collection in ZK if it doesn't exist for a given core. I'd missed transferring that part to the CloudConfigSetService. Working up a patch now.
          Hide
          Alan Woodward added a comment -

          Got it. ConfigSolr.createConfigSetService() checks the zkHost parameter to see if we're in cloud mode, and if we are, then returns a CloudConfigSetService. But in this case we're not passing a zkHost sysprop, we're passing zkRun instead. So it picks a default service instead, and uses a SolrResourceLoader based on the standard config, which means that the collection node never gets created.

          Show
          Alan Woodward added a comment - Got it. ConfigSolr.createConfigSetService() checks the zkHost parameter to see if we're in cloud mode, and if we are, then returns a CloudConfigSetService. But in this case we're not passing a zkHost sysprop, we're passing zkRun instead. So it picks a default service instead, and uses a SolrResourceLoader based on the standard config, which means that the collection node never gets created.
          Hide
          ASF subversion and git services added a comment -

          Commit 1582839 from Alan Woodward in branch 'dev/trunk'
          [ https://svn.apache.org/r1582839 ]

          SOLR-4478: Use CloudConfigSetService with embedded zk server

          Show
          ASF subversion and git services added a comment - Commit 1582839 from Alan Woodward in branch 'dev/trunk' [ https://svn.apache.org/r1582839 ] SOLR-4478 : Use CloudConfigSetService with embedded zk server
          Hide
          ASF subversion and git services added a comment -

          Commit 1582840 from Alan Woodward in branch 'dev/branches/branch_4x'
          [ https://svn.apache.org/r1582840 ]

          SOLR-4478: Use CloudConfigSetService with embedded zk server

          Show
          ASF subversion and git services added a comment - Commit 1582840 from Alan Woodward in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1582840 ] SOLR-4478 : Use CloudConfigSetService with embedded zk server
          Hide
          Shalin Shekhar Mangar added a comment -

          Thanks Alan!

          I guess the one time it succeeded for me was because I hadn't cleared my zookeeper directory

          Show
          Shalin Shekhar Mangar added a comment - Thanks Alan! I guess the one time it succeeded for me was because I hadn't cleared my zookeeper directory
          Hide
          Erick Erickson added a comment -

          Alan Woodward Can this be closed then? I'm also thinking that SOLR-4779 should just be closed as "won't fix" since I don't see a good reason to deprecate shareSchema. The hope was that we could share everything in a config set, but as I remember sharing solrconfig was "fraught". It seems to me that if we want to go farther down the sharing route thing, we need to use some other sharing model than piecemeal....

          Thoughts?

          Show
          Erick Erickson added a comment - Alan Woodward Can this be closed then? I'm also thinking that SOLR-4779 should just be closed as "won't fix" since I don't see a good reason to deprecate shareSchema. The hope was that we could share everything in a config set, but as I remember sharing solrconfig was "fraught". It seems to me that if we want to go farther down the sharing route thing, we need to use some other sharing model than piecemeal.... Thoughts?
          Hide
          Uwe Schindler added a comment -

          Move issue to Solr 4.9.

          Show
          Uwe Schindler added a comment - Move issue to Solr 4.9.

            People

            • Assignee:
              Alan Woodward
              Reporter:
              Erick Erickson
            • Votes:
              5 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

              • Created:
                Updated:

                Development