Solr
  1. Solr
  2. SOLR-336

Allow setting solr.data.dir with JNDI

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: update
    • Labels:
      None

      Description

      I would like to be able set the solr.data.dir with JNDI, as I can for the solr.home property. Currently the data directory is only looked up in the $

      {solr.home}

      /conf/solrconfig.xml file, or as a parameter passed into the SolrCore(String, IndexSchema) constructor.

      This allows more options for setting the data directory, such as from within a Servlet container's Context fragment.

        Issue Links

          Activity

          Erick Erickson made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Won't Fix [ 2 ]
          Hide
          Erick Erickson added a comment -

          Cleaning up old JIRAs, re-open if necessary.

          Show
          Erick Erickson added a comment - Cleaning up old JIRAs, re-open if necessary.
          Hide
          Mo Chen added a comment - - edited

          > Shalin Shekhar Mangar commented on SOLR-336:
          > --------------------------------------------
          >
          > Why should the dataDir be a special thing? Why not support JNDI values for all properties implicitly? We currently support two kinds of properties, the ones defined in solr.xml and the environment variables. As a logical extension, we can add JNDI variables too so that we don't need to duplicate JNDI properties again in solr.xml.

          Er, in fact I don't think dataDir is a special thing... I think any JNDI
          configuration is special thing, if it has the same meaning with a system
          property.
          The dataDir is just a example — I added it just because it's the only
          specificated one in the wiki document.

          But I opposite to the concept of refering properties and JNDI values in
          different syntax...
          for example, if <dataDir>$

          {java:comp/env/solr/home}

          /data</dataDir> and
          <dataDir>$

          {solr.solr.home}

          /data</dataDir> are both supported
          which is the one I should put into the solrconfig.xml?
          There might be two solr servers using this configuration... one of them has
          a JNDI home, and the other one is started from command line.

          I think the problem is what we needed in configuration files is not a
          parameter from somewhere, but a server's status in fact.
          The server can be installed with a console parameter, with a JNDI context
          file, or with hardcode in web.xml, it's doesn't matter.
          I just want to know where the server's home is.

          So, we either map both the JNDI value and system property into a same system
          status variable, or just use properties to do this, and map JNDI values to
          it.
          That's why I only enabled JNDI lookup in property values.
          If you think hardcode mapping into Config.java is to ugly, maybe a mapping
          properties file in solr home dir will be a better choise?

          Anyway, thanks for replying

          > I guess the property syntax will need to be changed a bit because ':' is used for default values. We can either escape the ':' in JNDI name or we can make the DOMUtil.substituteProperty method aware of this change.

          Yes, I totally agree with this.

          Show
          Mo Chen added a comment - - edited > Shalin Shekhar Mangar commented on SOLR-336 : > -------------------------------------------- > > Why should the dataDir be a special thing? Why not support JNDI values for all properties implicitly? We currently support two kinds of properties, the ones defined in solr.xml and the environment variables. As a logical extension, we can add JNDI variables too so that we don't need to duplicate JNDI properties again in solr.xml. Er, in fact I don't think dataDir is a special thing... I think any JNDI configuration is special thing, if it has the same meaning with a system property. The dataDir is just a example — I added it just because it's the only specificated one in the wiki document. But I opposite to the concept of refering properties and JNDI values in different syntax... for example, if <dataDir>$ {java:comp/env/solr/home} /data</dataDir> and <dataDir>$ {solr.solr.home} /data</dataDir> are both supported which is the one I should put into the solrconfig.xml? There might be two solr servers using this configuration... one of them has a JNDI home, and the other one is started from command line. I think the problem is what we needed in configuration files is not a parameter from somewhere, but a server's status in fact. The server can be installed with a console parameter, with a JNDI context file, or with hardcode in web.xml, it's doesn't matter. I just want to know where the server's home is. So, we either map both the JNDI value and system property into a same system status variable, or just use properties to do this, and map JNDI values to it. That's why I only enabled JNDI lookup in property values. If you think hardcode mapping into Config.java is to ugly, maybe a mapping properties file in solr home dir will be a better choise? Anyway, thanks for replying > I guess the property syntax will need to be changed a bit because ':' is used for default values. We can either escape the ':' in JNDI name or we can make the DOMUtil.substituteProperty method aware of this change. Yes, I totally agree with this.
          Mo Chen made changes -
          Link This issue is part of SOLR-1007 [ SOLR-1007 ]
          Hide
          Shalin Shekhar Mangar added a comment -

          Why should the dataDir be a special thing? Why not support JNDI values for all properties implicitly? We currently support two kinds of properties, the ones defined in solr.xml and the environment variables. As a logical extension, we can add JNDI variables too so that we don't need to duplicate JNDI properties again in solr.xml.

          I guess the property syntax will need to be changed a bit because ':' is used for default values. We can either escape the ':' in JNDI name or we can make the DOMUtil.substituteProperty method aware of this change. If we go this way, I think the only piece of code which would need to be changed in DOMUtil.substituteProperty method.

          Show
          Shalin Shekhar Mangar added a comment - Why should the dataDir be a special thing? Why not support JNDI values for all properties implicitly? We currently support two kinds of properties, the ones defined in solr.xml and the environment variables. As a logical extension, we can add JNDI variables too so that we don't need to duplicate JNDI properties again in solr.xml. I guess the property syntax will need to be changed a bit because ':' is used for default values. We can either escape the ':' in JNDI name or we can make the DOMUtil.substituteProperty method aware of this change. If we go this way, I think the only piece of code which would need to be changed in DOMUtil.substituteProperty method.
          Mo Chen made changes -
          Attachment solr-336-jndi-props.patch [ 12399431 ]
          Mo Chen made changes -
          Attachment solr-336-jndi-props.patch [ 12399431 ]
          Hide
          Mo Chen added a comment - - edited

          convert JNDI name into properties by using <property> element in solrconfig.xml

          I got the same problem yesterday... and this is a solution following the concept of converting JNDI into properties.

          The plan is to implement solr.xml's <property> element in solrconfig.xml, and append JNDI supporting to it.
          3 changes for the implementation:

          • Move private method readProperties(Config cfg, Node node) from CoreContainer to Config,
            use it to read <property> elements in solrconfig.xml during the contruction of Config,
            and overwrite coreProperties before the process DOMUtil.substituteProperties.

            this ensure the mapped JNDI values only take effect in solrconfig.xml, but not the system scope
            the moved method got a Config for the first argument already... so I think that will be ok

          • Check property values and replace all 'java:comp/env/xxx' format value into real JNDI value.

            I write ${:java:comp/env/xxx} for the JNDI ref string... not very good. better solution?

          • Append some default jndi mapping. In fact only one. The "solr/home" and "solr.solr.home"...

            the priority: <property> in solrconfig -> default jndi mapping -> core properties

          Now you can got JNDI values in solrconfig.xml like this:

          <property name="solr.solr.home" value="java:comp/env/solr/home" />
          <dataDir>${solr.solr.home:./solr}/data</dataDir>
          

          (Or just the second line, this mapping has been setted as default)

          Show
          Mo Chen added a comment - - edited convert JNDI name into properties by using <property> element in solrconfig.xml I got the same problem yesterday... and this is a solution following the concept of converting JNDI into properties. The plan is to implement solr.xml 's <property> element in solrconfig.xml , and append JNDI supporting to it. 3 changes for the implementation: Move private method readProperties(Config cfg, Node node) from CoreContainer to Config , use it to read <property> elements in solrconfig.xml during the contruction of Config , and overwrite coreProperties before the process DOMUtil.substituteProperties . this ensure the mapped JNDI values only take effect in solrconfig.xml, but not the system scope the moved method got a Config for the first argument already... so I think that will be ok Check property values and replace all 'java:comp/env/xxx' format value into real JNDI value. I write ${:java:comp/env/xxx} for the JNDI ref string... not very good. better solution ? Append some default jndi mapping. In fact only one. The "solr/home" and "solr.solr.home"... the priority: <property> in solrconfig -> default jndi mapping -> core properties Now you can got JNDI values in solrconfig.xml like this: <property name= "solr.solr.home" value= "java:comp/env/solr/home" /> <dataDir> ${solr.solr.home:./solr}/data </dataDir> (Or just the second line, this mapping has been setted as default)
          Hide
          Stu Hood added a comment -

          I like your idea about system properties being imported for use in solrconfig.xml, perhaps I'll get a chance to implement that sometime soon to replace this patch.

          Thanks

          Show
          Stu Hood added a comment - I like your idea about system properties being imported for use in solrconfig.xml, perhaps I'll get a chance to implement that sometime soon to replace this patch. Thanks
          Hide
          Hoss Man added a comment -

          I'm opposed to the concept behind this patch for the same reasons i was opposed to adding a specific system property for the data dir...

          http://www.nabble.com/Re%3A-finer-granularity-of-configuration-p8894861.html

          perhaps instead of adding piece meal JNDI names for various things normally specified in the solrconfig.xml, we should take advantage of the existing code that supports using system properties as variables in the solrconfig.xml and write something that will generically convert any comp/env/solr/* JNDI pairs found into solr.* system properties before parsing the config? that way people can define any JNDI name they want (as long as it starts with solr) and then refer to those names in the solrconfig.xml just like they can with system properties right now.

          I don't know much about JNDI to know how feasible it is to get a list of all names matching a prefix, but i'm assuming it's possible (even if it's inefficient, we only need to do it once on startup)

          Show
          Hoss Man added a comment - I'm opposed to the concept behind this patch for the same reasons i was opposed to adding a specific system property for the data dir... http://www.nabble.com/Re%3A-finer-granularity-of-configuration-p8894861.html perhaps instead of adding piece meal JNDI names for various things normally specified in the solrconfig.xml, we should take advantage of the existing code that supports using system properties as variables in the solrconfig.xml and write something that will generically convert any comp/env/solr/* JNDI pairs found into solr.* system properties before parsing the config? that way people can define any JNDI name they want (as long as it starts with solr) and then refer to those names in the solrconfig.xml just like they can with system properties right now. I don't know much about JNDI to know how feasible it is to get a list of all names matching a prefix, but i'm assuming it's possible (even if it's inefficient, we only need to do it once on startup)
          Stu Hood made changes -
          Field Original Value New Value
          Attachment jndi-data-dir.patch [ 12363711 ]
          Hide
          Stu Hood added a comment -

          Here is a patch to allow the data directory to be set either by JNDI or a system property.

          I wasn't sure whether to put the logic in core/SolrCore or core/Config, but since core/Config already imported the necessary libraries for JNDI, and I added it there.

          Show
          Stu Hood added a comment - Here is a patch to allow the data directory to be set either by JNDI or a system property. I wasn't sure whether to put the logic in core/SolrCore or core/Config, but since core/Config already imported the necessary libraries for JNDI, and I added it there.
          Stu Hood created issue -

            People

            • Assignee:
              Unassigned
              Reporter:
              Stu Hood
            • Votes:
              2 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development