Solr
  1. Solr
  2. SOLR-3182

If there is only one core, let it be the default without specifying a default in solr.xml

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: 3.6, 4.0-ALPHA
    • Fix Version/s: None
    • Component/s: multicore
    • Labels:

      Description

      Our particular need for this is as follows. We operate in a sharded environment with one core per server. Each shard also acts as a collator. We want to use a hardware load balancer to choose which shard will do the collation for each query. But in order to do that, each server's single core would have to carry the same name so that it could be accessed by the same url across servers. However we name the cores by their shard number (query0,query1,...) because it parallels with the way we name our indexing/master cores (index0, index1,...). This naming convention also gives us the flexibility of moving to a multicore environment in the future without having to rename the cores, although admittedly that would complicate load balancing.

      In a system with a large number of shards and the anticipation of adding more going forward, setting a defaultCoreName attribute in each solr.xml file becomes inconvenient, especially since there is no Solr admin API for setting defaultCoreName. It would have to be done by hand or with some automated tool we would write in house. Even if there were an API, logically it seems unnecessary to have to declare the only core to be the default.

      Fortunately this behavior can be implemented with the following simple patch:

      Index: solr/core/src/java/org/apache/solr/core/CoreContainer.java
      ===================================================================
      --- solr/core/src/java/org/apache/solr/core/CoreContainer.java	(revision 1295229)
      +++ solr/core/src/java/org/apache/solr/core/CoreContainer.java	(working copy)
      @@ -870,6 +870,10 @@
         }
       
         private String checkDefault(String name) {
      +    // if there is only one core, let it be the default without specifying a default in solr.xml
      +    if (defaultCoreName.trim().length() == 0 && name.trim().length() == 0 && cores.size() == 1) {
      +      return cores.values().iterator().next().getName();
      +    }
           return name.length() == 0  || defaultCoreName.equals(name) || name.trim().length() == 0 ? "" : name;
         } 
      

        Activity

        Hide
        Mark Miller added a comment -

        Sounds like they where using it like an alias? We have alias support in SolrCloud now, but the single node alias stuff was previously deprecated.

        Show
        Mark Miller added a comment - Sounds like they where using it like an alias? We have alias support in SolrCloud now, but the single node alias stuff was previously deprecated.
        Hide
        Shawn Heisey added a comment -

        Russell Black, I don't think the defaultCoreName attribute was really intended to be a long-term thing. It bridged the gap between the single-core days and the switch to multi-core as the example default. Solr 5.0 will be fully multicore, I expect that it will no longer be possible to configure a single-core setup.

        The fact that beginner Solr documentation continues even now to use the http://server:port/solr base URL is going to be a problem in the future when URLs that don't include a core/collection name will cease to work. I'm actually surprised that they are still supported in 4.x.

        Show
        Shawn Heisey added a comment - Russell Black , I don't think the defaultCoreName attribute was really intended to be a long-term thing. It bridged the gap between the single-core days and the switch to multi-core as the example default. Solr 5.0 will be fully multicore, I expect that it will no longer be possible to configure a single-core setup. The fact that beginner Solr documentation continues even now to use the http://server:port/solr base URL is going to be a problem in the future when URLs that don't include a core/collection name will cease to work. I'm actually surprised that they are still supported in 4.x.
        Hide
        Russell Black added a comment -

        That will likely force us to re-architect our load balancing system (described in the ticket summary above). Is there a ticket or a thread where I can read more about this change?

        Show
        Russell Black added a comment - That will likely force us to re-architect our load balancing system (described in the ticket summary above). Is there a ticket or a thread where I can read more about this change?
        Hide
        Mark Miller added a comment -

        default core is going away

        Show
        Mark Miller added a comment - default core is going away
        Hide
        Otis Gospodnetic added a comment -

        Erick Erickson is working on changing a ton around solr.xml, so maybe this no longer even applies?

        Show
        Otis Gospodnetic added a comment - Erick Erickson is working on changing a ton around solr.xml, so maybe this no longer even applies?
        Hide
        Russell Black added a comment -

        We have been using this patch in production now for two years. Would it be possible to get this into the 4.x code base?

        Show
        Russell Black added a comment - We have been using this patch in production now for two years. Would it be possible to get this into the 4.x code base?
        Hide
        Erick Erickson added a comment -

        Don't have time to get to this in 3.6, does someone else want to push this forward?

        Show
        Erick Erickson added a comment - Don't have time to get to this in 3.6, does someone else want to push this forward?
        Hide
        Erick Erickson added a comment -

        Do we want to carry this forward for 3.x? If so, we need to get consensus real soon now since there's a movement to cut a 3.6 (the last 3x release planned?) in 10 days or so.

        I've assigned it to myself just as a placeholder, if someone else wants to go ahead and take this, please feel free.

        Or does all the SolrCloud stuff make this something we don't want to deal with (in which case there never will be a 3.x way to handle this)...

        Or does it make sense to do this for 3.6 and not try to move it into trunk?

        Show
        Erick Erickson added a comment - Do we want to carry this forward for 3.x? If so, we need to get consensus real soon now since there's a movement to cut a 3.6 (the last 3x release planned?) in 10 days or so. I've assigned it to myself just as a placeholder, if someone else wants to go ahead and take this, please feel free. Or does all the SolrCloud stuff make this something we don't want to deal with (in which case there never will be a 3.x way to handle this)... Or does it make sense to do this for 3.6 and not try to move it into trunk?
        Hide
        Russell Black added a comment - - edited

        Jan, that is an interesting idea and is similar to the one Yonik proposed earlier. I am happy with your proposal as it would address our situation just fine.

        That said, you could experience a type of inconsistent behavior with a $1 solution as well. Let's say I have two cores and have defaultCoreName=$1. Then the first core gets deleted. Anyone who was making requests to the default core(/solr/select) will suddenly find that they are getting different data because they are hitting a new core. From this perspective, the "magic" behavior would seem to be no worse than the inconsistency with the $1 solution. It's possible I am missing something here.

        As far as the backwards compatibility of my proposed solution, old solr.xmls will continue to operate as they always have, the only difference being that single cores will be available via an additional url. That is, single cores will continue to be accessible by their name.

        I am willing to build the patch for whatever solution the committers agree on.

        Show
        Russell Black added a comment - - edited Jan, that is an interesting idea and is similar to the one Yonik proposed earlier. I am happy with your proposal as it would address our situation just fine. That said, you could experience a type of inconsistent behavior with a $1 solution as well. Let's say I have two cores and have defaultCoreName=$1 . Then the first core gets deleted. Anyone who was making requests to the default core(/solr/select) will suddenly find that they are getting different data because they are hitting a new core. From this perspective, the "magic" behavior would seem to be no worse than the inconsistency with the $1 solution. It's possible I am missing something here. As far as the backwards compatibility of my proposed solution, old solr.xmls will continue to operate as they always have, the only difference being that single cores will be available via an additional url. That is, single cores will continue to be accessible by their name. I am willing to build the patch for whatever solution the committers agree on.
        Hide
        Jan Høydahl added a comment -

        I'd prefer to avoid such "magic" inconsistent behavior.

        Instead I propose that if you specify defaultCoreName="$1" it will choose the first core in the list $2 the second etc. This will keep old solr.xml's backward compatible. Sample:

          <cores adminPath="/admin/cores" defaultCoreName="$1">
            <core name="myCore" instanceDir="myCore" />
          </cores>
        
        Show
        Jan Høydahl added a comment - I'd prefer to avoid such "magic" inconsistent behavior. Instead I propose that if you specify defaultCoreName="$1" it will choose the first core in the list $2 the second etc. This will keep old solr.xml's backward compatible. Sample: <cores adminPath= "/admin/cores" defaultCoreName= "$1" > <core name= "myCore" instanceDir= "myCore" /> </cores>
        Hide
        Russell Black added a comment - - edited

        Perhaps the confusion you point out could be mitigated with some messaging, something like this:

        private String checkDefault(String name) {
          // if there is only one core, let it be the default without specifying a default in solr.xml
          if (StringUtils.isBlank(defaultCoreName) && StringUtils.isBlank(name.trim())) {
            if (cores.size() == 1) {
              return cores.values().iterator().next().getName();
            }
            else {
              throw new SolrException( SolrException.ErrorCode.BAD_REQUEST, "This server hosts multiple cores, " +
              		"but no core name was supplied in the request and no defaultCoreName was declared in solr.xml" );
            }
          }
          return StringUtils.isBlank(name) || defaultCoreName.equals(name) ? "" : name;
        } 
        
        Show
        Russell Black added a comment - - edited Perhaps the confusion you point out could be mitigated with some messaging, something like this: private String checkDefault( String name) { // if there is only one core, let it be the default without specifying a default in solr.xml if (StringUtils.isBlank(defaultCoreName) && StringUtils.isBlank(name.trim())) { if (cores.size() == 1) { return cores.values().iterator().next().getName(); } else { throw new SolrException( SolrException.ErrorCode.BAD_REQUEST, "This server hosts multiple cores, " + "but no core name was supplied in the request and no defaultCoreName was declared in solr.xml" ); } } return StringUtils.isBlank(name) || defaultCoreName.equals(name) ? "" : name; }
        Hide
        Russell Black added a comment - - edited

        But then the reverse case could also be jarring when someone is able to access a core by virtue of it being default, and then that ability is suddenly removed when a new core is created.

        Agreed. An application developer would need to understand the implications of adding another core and plan accordingly in his application, or else explicitly declare a defaultCoreName in solr.xml.

        Show
        Russell Black added a comment - - edited But then the reverse case could also be jarring when someone is able to access a core by virtue of it being default, and then that ability is suddenly removed when a new core is created. Agreed. An application developer would need to understand the implications of adding another core and plan accordingly in his application, or else explicitly declare a defaultCoreName in solr.xml.
        Hide
        Yonik Seeley added a comment -

        It simply becomes accessible by an additional URL when it becomes the only core.

        But then the reverse case could also be jarring when someone is able to access a core by virtue of it being default, and then that ability is suddenly removed when a new core is created.

        Show
        Yonik Seeley added a comment - It simply becomes accessible by an additional URL when it becomes the only core. But then the reverse case could also be jarring when someone is able to access a core by virtue of it being default, and then that ability is suddenly removed when a new core is created.
        Hide
        Mark Miller added a comment -

        Seems like a reasonable idea. I'll try to play with the patch a little when I get a free moment.

        Show
        Mark Miller added a comment - Seems like a reasonable idea. I'll try to play with the patch a little when I get a free moment.
        Hide
        Russell Black added a comment -

        If you start with two cores open (and maybe even no default core), then close one core and all of a sudden the other becomes the default?

        I have thought about that situation. It is true that going from two cores to one would result in the remaining core becoming the default. However, this doesn't change the way the server presents itself to an application that expects a multi-core system, since the remaining core continues to be accessible by the same URL as it was before. It simply becomes accessible by an additional URL when it becomes the only core.

        Thanks for considering this patch.

        Show
        Russell Black added a comment - If you start with two cores open (and maybe even no default core), then close one core and all of a sudden the other becomes the default? I have thought about that situation. It is true that going from two cores to one would result in the remaining core becoming the default. However, this doesn't change the way the server presents itself to an application that expects a multi-core system, since the remaining core continues to be accessible by the same URL as it was before. It simply becomes accessible by an additional URL when it becomes the only core. Thanks for considering this patch.
        Hide
        Yonik Seeley added a comment -

        Other than that, does anyone object to committing this?

        I haven't thought through the implications, but it could be dicey for multi-core use.
        If you start with two cores open (and maybe even no default core), then close one core and all of a sudden the other becomes the default?

        Perhaps this need for a default core could instead be generalized... some sort of config option that made the default the first active core (sorted by core name)?

        Show
        Yonik Seeley added a comment - Other than that, does anyone object to committing this? I haven't thought through the implications, but it could be dicey for multi-core use. If you start with two cores open (and maybe even no default core), then close one core and all of a sudden the other becomes the default? Perhaps this need for a default core could instead be generalized... some sort of config option that made the default the first active core (sorted by core name)?
        Hide
        Erick Erickson added a comment -

        Hmmm, there's no guarantee that the incoming name is not null, so wouldn't
        StringUtils.isBlank() (commons-lang is available) be a better choice for testing name? defaultCoreName is initialized to "", but it doesn't seem like it would hurt there either...

        Other than that, does anyone object to committing this? I don't see how this bit could generate an NPE given that the size is guaranteed to be > 0
        cores.values().iterator().next().getName()

        Show
        Erick Erickson added a comment - Hmmm, there's no guarantee that the incoming name is not null, so wouldn't StringUtils.isBlank() (commons-lang is available) be a better choice for testing name? defaultCoreName is initialized to "", but it doesn't seem like it would hurt there either... Other than that, does anyone object to committing this? I don't see how this bit could generate an NPE given that the size is guaranteed to be > 0 cores.values().iterator().next().getName()

          People

          • Assignee:
            Unassigned
            Reporter:
            Russell Black
          • Votes:
            2 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:

              Time Tracking

              Estimated:
              Original Estimate - 10m
              10m
              Remaining:
              Remaining Estimate - 10m
              10m
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development