Solr
  1. Solr
  2. SOLR-791

Allow to submit config and schema when creating a new core

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: 1.3
    • Fix Version/s: None
    • Component/s: clients - java
    • Labels:
      None

      Description

      Currently it's possible to create cores "remotely" via SolrJ.

      CoreAdminRequest.createCore("acore", "acoreinstancedir", adminServer);
      

      However, this process is incomplete because I need to manually log onto the remote server and place a configuration file as well as a schema file into the conf/ folder in the acoreinstancedir/. It would be great it I can simply submit those files together with the create core request.

        Issue Links

          Activity

          Hide
          Hoss Man added a comment -

          This doesn't seem like something that should be a built in feature of Solr ... userswho want to be able to remotely install config files should use WebDAV or other tools designed for such a purpose.

          Show
          Hoss Man added a comment - This doesn't seem like something that should be a built in feature of Solr ... userswho want to be able to remotely install config files should use WebDAV or other tools designed for such a purpose.
          Hide
          Gunnar Wagenknecht added a comment -

          Hmm, but why is it possible to setup cores remotely? It sounds unreasonable to install, setup and maintain a separate systemto allow WebDAV just for uploading configuration files.

          It would be easier if I can make a single POSTrequest to setup & create a new core. This would be a single, atomic request and avoids a second system which helps keeping the operational costs under control.

          Show
          Gunnar Wagenknecht added a comment - Hmm, but why is it possible to setup cores remotely? It sounds unreasonable to install, setup and maintain a separate systemto allow WebDAV just for uploading configuration files. It would be easier if I can make a single POSTrequest to setup & create a new core. This would be a single, atomic request and avoids a second system which helps keeping the operational costs under control.
          Hide
          Vladimir Manolov added a comment -

          This feature would be really helpful, if one does not have access to the filesystem of a production environment, which is often the case in large companies, which have some deployment processes. Adding WebDAV to existing environments is often difficult, since there is some kind of established infrastructure in the enterprise and the company is not flexible enough to allow specific hosting solutions, which would include WebDAV.

          Show
          Vladimir Manolov added a comment - This feature would be really helpful, if one does not have access to the filesystem of a production environment, which is often the case in large companies, which have some deployment processes. Adding WebDAV to existing environments is often difficult, since there is some kind of established infrastructure in the enterprise and the company is not flexible enough to allow specific hosting solutions, which would include WebDAV.
          Hide
          Erick Erickson added a comment -

          To actually work, you'd need to upload essentially all of the conf directory, as well as any sub-directories. For instance, what about stopwords.txt? protwords.txt? the velocity and xslt subdirs? dictionary files? elevate.xml? currency.xml? Simply uploading the schema.xml and solrconfig.xml files will fail if any filter references, say, stopwords.txt.

          Show
          Erick Erickson added a comment - To actually work, you'd need to upload essentially all of the conf directory, as well as any sub-directories. For instance, what about stopwords.txt? protwords.txt? the velocity and xslt subdirs? dictionary files? elevate.xml? currency.xml? Simply uploading the schema.xml and solrconfig.xml files will fail if any filter references, say, stopwords.txt.
          Hide
          Hooman Mozaffari added a comment -

          As an alternative you can extend CoreAdminHandler class, overwrite handleCustomAction method and implement the extra functionality.

          In Solr 4.4.0 and later make sure you have introduced your custom core admin handler by setting up the following attributes in solr.xml:

          <str name="sharedLib">[location of your shared libraries including the jar file containing your new admin handler class]</str>
          <str name="adminHandler">[fully qualified name of your new admin handler class]</str>
          
          Show
          Hooman Mozaffari added a comment - As an alternative you can extend CoreAdminHandler class, overwrite handleCustomAction method and implement the extra functionality. In Solr 4.4.0 and later make sure you have introduced your custom core admin handler by setting up the following attributes in solr.xml : <str name= "sharedLib" > [location of your shared libraries including the jar file containing your new admin handler class] </str> <str name= "adminHandler" > [fully qualified name of your new admin handler class] </str>
          Hide
          Erick Erickson added a comment -

          This will be superceded by named config sets, a separate JIRA.

          Show
          Erick Erickson added a comment - This will be superceded by named config sets, a separate JIRA.
          Hide
          Alexey Serba added a comment -

          This will be superceded by named config sets, a separate JIRA

          Erick Erickson, could you please link to that issue? If you meant SOLR-5287 then I believe this API will be available only for existing cores/collections, right?

          Another idea is that Solr should have default configs (example?) bundled into a jar (so this is part of the installation). These configs will be used at core/collection creation time if you omit instanceDir/configSet parameters. WDYT?

          Show
          Alexey Serba added a comment - This will be superceded by named config sets, a separate JIRA Erick Erickson , could you please link to that issue? If you meant SOLR-5287 then I believe this API will be available only for existing cores/collections, right? Another idea is that Solr should have default configs (example?) bundled into a jar (so this is part of the installation). These configs will be used at core/collection creation time if you omit instanceDir/configSet parameters. WDYT?
          Hide
          Mark Miller added a comment -

          Another idea is that Solr should have default configs (example?) bundled into a jar (so this is part of the installation). These configs will be used at core/collection creation time if you omit instanceDir/configSet parameters. WDYT?

          +1 - I brought this up on the mailing list the other day. You should be able to do more as you work with a database - you get a bunch of starting settings for your collection even with no work, and you can tweak them as necessary after the fact. Or of course you could make new template collection sets or tweak them up front, but that should be a choice rather than kind of forced in your face.

          Show
          Mark Miller added a comment - Another idea is that Solr should have default configs (example?) bundled into a jar (so this is part of the installation). These configs will be used at core/collection creation time if you omit instanceDir/configSet parameters. WDYT? +1 - I brought this up on the mailing list the other day. You should be able to do more as you work with a database - you get a bunch of starting settings for your collection even with no work, and you can tweak them as necessary after the fact. Or of course you could make new template collection sets or tweak them up front, but that should be a choice rather than kind of forced in your face.
          Hide
          Erick Erickson added a comment -

          [~aserba] Not SOLR-5287, but SOLR-4779. That said, though, SOLR-5287 would allow building up from an absolutely minimal skeleton, see solrconfig-minimal and schema-tiny in the test code. The problem here would just be that that sometime during the complex process of making a full-blown set of configs, I'll mess it up sometime and have to manually edit the mistakes just like before. If the core doesn't load, you can't use the new editing code. If the core fails to load, you can't select from the admin UI.

          Show
          Erick Erickson added a comment - [~aserba] Not SOLR-5287 , but SOLR-4779 . That said, though, SOLR-5287 would allow building up from an absolutely minimal skeleton, see solrconfig-minimal and schema-tiny in the test code. The problem here would just be that that sometime during the complex process of making a full-blown set of configs, I'll mess it up sometime and have to manually edit the mistakes just like before. If the core doesn't load, you can't use the new editing code. If the core fails to load, you can't select from the admin UI.
          Hide
          Erick Erickson added a comment -

          4779 should probably make this unnecessary. SOLR-5287 is also related, although somewhat tangentially.

          Show
          Erick Erickson added a comment - 4779 should probably make this unnecessary. SOLR-5287 is also related, although somewhat tangentially.
          Hide
          Mark Miller added a comment -

          If the core doesn't load, you can't use the new editing code. If the core fails to load, you can't select from the admin UI.

          We should probably try and allow some sort of sanity check - perhaps by loading up a tmp core or something...if it fails, we don't engage the change.

          Show
          Mark Miller added a comment - If the core doesn't load, you can't use the new editing code. If the core fails to load, you can't select from the admin UI. We should probably try and allow some sort of sanity check - perhaps by loading up a tmp core or something...if it fails, we don't engage the change.
          Hide
          Mark Miller added a comment -

          Although I guess you are talking about submitting the configs with the core create call? I think thats fine if it fails - it should just fail atomically - all or nothing. Then you change your configs and try again based on the errors.

          Show
          Mark Miller added a comment - Although I guess you are talking about submitting the configs with the core create call? I think thats fine if it fails - it should just fail atomically - all or nothing. Then you change your configs and try again based on the errors.
          Hide
          Erick Erickson added a comment -

          bq: We should probably try and allow some sort of sanity check - perhaps by loading up a tmp core or something...if it fails, we don't engage the change.

          Yeah, I've been deferring the hardening of error cases, figuring it would be good to get the basic functionality in place first then figure out how to harden. "Progress not perfection" and all that. Hadn't really though about this option, but it makes lots of sense and lets the system do the work. I'll raise a JIRA for this, thanks for the idea!

          Show
          Erick Erickson added a comment - bq: We should probably try and allow some sort of sanity check - perhaps by loading up a tmp core or something...if it fails, we don't engage the change. Yeah, I've been deferring the hardening of error cases, figuring it would be good to get the basic functionality in place first then figure out how to harden. "Progress not perfection" and all that. Hadn't really though about this option, but it makes lots of sense and lets the system do the work. I'll raise a JIRA for this, thanks for the idea!

            People

            • Assignee:
              Unassigned
              Reporter:
              Gunnar Wagenknecht
            • Votes:
              7 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development