Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.0, 6.0
    • Component/s: None
    • Labels:
      None

      Description

      There are a bunch of properties in solrconfig.xml which users want to edit. We will attack them first

      These properties will be persisted to a separate file called config.json (or whatever file). Instead of saving in the same format we will have well known properties which users can directly edit

      updateHandler.autoCommit.maxDocs
      query.filterCache.initialSize
      

      The api will be modeled around the bulk schema API

      curl http://localhost:8983/solr/collection1/config -H 'Content-type:application/json'  -d '{
      "set-property" : {"updateHandler.autoCommit.maxDocs":5},
      "unset-property": "updateHandler.autoCommit.maxDocs"
      }'
      
      //or use this to set ${mypropname} values
      curl http://localhost:8983/solr/collection1/config -H 'Content-type:application/json'  -d '{
      "set-user-property" : {"mypropname":"my_prop_val"},
      "unset-user-property":{"mypropname"}
      }'
      
      

      The values stored in the config.json will always take precedence and will be applied after loading solrconfig.xml.

      • An http GET on /config path will give the real config that is applied .
      • An http GET of/config/overlay gives out the content of the configOverlay.json
      • /config/<component-name> gives only the child of the same name from /config . example: /config/requestHandler , /config/searchComponent
      1. SOLR-6533.patch
        34 kB
        Noble Paul
      2. SOLR-6533.patch
        35 kB
        Noble Paul
      3. SOLR-6533.patch
        34 kB
        Noble Paul
      4. SOLR-6533.patch
        97 kB
        Noble Paul
      5. SOLR-6533.patch
        99 kB
        Noble Paul
      6. SOLR-6533.patch
        92 kB
        Noble Paul
      7. SOLR-6533.patch
        76 kB
        Noble Paul
      8. SOLR-6533.patch
        55 kB
        Noble Paul
      9. SOLR-6533.patch
        44 kB
        Noble Paul
      10. SOLR-6533.patch
        38 kB
        Noble Paul
      11. SOLR-6533.patch
        23 kB
        Noble Paul

        Issue Links

          Activity

          Hide
          Noble Paul added a comment -

          First cut. No ZK support or No testcases yet

          Show
          Noble Paul added a comment - First cut. No ZK support or No testcases yet
          Hide
          Noble Paul added a comment -

          second cut . ZK support added

          Show
          Noble Paul added a comment - second cut . ZK support added
          Hide
          Noble Paul added a comment -

          new approach. Added testcases. pretty close to the final shape

          Show
          Noble Paul added a comment - new approach. Added testcases. pretty close to the final shape
          Hide
          Tomás Fernández Löbbe added a comment -

          Can this handler be enabled/disabled? What about extended? For example, if I have a custom component that has init params, could I make it so that I can modify those too?
          In the code, I see some exceptions that don't log or re-throw, we may be losing some information there.
          And a question, what's "SET_USER_PROPERTY"

          Show
          Tomás Fernández Löbbe added a comment - Can this handler be enabled/disabled? What about extended? For example, if I have a custom component that has init params, could I make it so that I can modify those too? In the code, I see some exceptions that don't log or re-throw, we may be losing some information there. And a question, what's "SET_USER_PROPERTY"
          Hide
          Noble Paul added a comment -

          Can this handler be enabled/disabled?

          No. that is not possible

          In the code, I see some exceptions that don't log or re-throw, we may be losing some information there.

          This patch is not yet final . those will bed fixed anyway

          what's "SET_USER_PROPERTY"

          This will be implemented in the next patch
          The set-property automatically maps to certain well-known values in solrconfig.xml . If you have a property defined in solrconfig.xml using the $

          {variable-name:default_value}

          format, you can use the commant "set-user-property" :

          {"variable-name":"vaiable-value"}

          to control it in run-time

          "initParams" are likely to be targeted in another issue (not yet created)

          Show
          Noble Paul added a comment - Can this handler be enabled/disabled? No. that is not possible In the code, I see some exceptions that don't log or re-throw, we may be losing some information there. This patch is not yet final . those will bed fixed anyway what's "SET_USER_PROPERTY" This will be implemented in the next patch The set-property automatically maps to certain well-known values in solrconfig.xml . If you have a property defined in solrconfig.xml using the $ {variable-name:default_value} format, you can use the commant "set-user-property" : {"variable-name":"vaiable-value"} to control it in run-time "initParams" are likely to be targeted in another issue (not yet created)
          Hide
          Noble Paul added a comment -

          All features done.

          Cleanup is left and of course more testcases.

          Please give your inputs

          Show
          Noble Paul added a comment - All features done. Cleanup is left and of course more testcases. Please give your inputs
          Hide
          Tomás Fernández Löbbe added a comment -

          Can this handler be enabled/disabled?

          No. that is not possible

          I think this should be supported. My understanding is that one can enable/disable the schama API from the solrconfig by setting the SchemaFactory. In addition, one could have it's own custom schema factory, right?

          if you have a property defined in solrconfig.xml using the ${variable-name:default_value} format, you can use the command "set-user-property"

          That sounds really useful

          Show
          Tomás Fernández Löbbe added a comment - Can this handler be enabled/disabled? No. that is not possible I think this should be supported. My understanding is that one can enable/disable the schama API from the solrconfig by setting the SchemaFactory. In addition, one could have it's own custom schema factory, right? if you have a property defined in solrconfig.xml using the ${variable-name:default_value} format, you can use the command "set-user-property" That sounds really useful
          Hide
          Noble Paul added a comment - - edited

          with a testcase for concurrent tests

          this patch includes a fix for SOLR-6654 as well

          Show
          Noble Paul added a comment - - edited with a testcase for concurrent tests this patch includes a fix for SOLR-6654 as well
          Hide
          Noble Paul added a comment -

          This is pretty close to a commit. Please give your comments

          Show
          Noble Paul added a comment - This is pretty close to a commit. Please give your comments
          Hide
          Tomás Fernández Löbbe added a comment -

          What about my previous comment? I think users should be able to enable/disable this feature.
          A couple of minor things:
          I saw some e.printStackTrace();//todo handle Exception that should probably be fixed before committing
          Also, there are many places where generics are used without parameters. I know there are lots of places in the current code when this is happening, but I think new code should always use parametrized generics when possible.
          There is some commented out code, should that be removed instead?

          Show
          Tomás Fernández Löbbe added a comment - What about my previous comment? I think users should be able to enable/disable this feature. A couple of minor things: I saw some e.printStackTrace();//todo handle Exception that should probably be fixed before committing Also, there are many places where generics are used without parameters. I know there are lots of places in the current code when this is happening, but I think new code should always use parametrized generics when possible. There is some commented out code, should that be removed instead?
          Hide
          Noble Paul added a comment -

          TODO and printStacktrace etc will be fixed before any checkin happens

          I think users should be able to enable/disable this feature.

          Yes that will be added as a system property

          Show
          Noble Paul added a comment - TODO and printStacktrace etc will be fixed before any checkin happens I think users should be able to enable/disable this feature. Yes that will be added as a system property
          Hide
          Noble Paul added a comment -

          all tests pass,
          added a command line option to disable config editing

          Show
          Noble Paul added a comment - all tests pass, added a command line option to disable config editing
          Hide
          ASF subversion and git services added a comment -

          Commit 1636862 from Noble Paul in branch 'dev/trunk'
          [ https://svn.apache.org/r1636862 ]

          SOLR-6533,SOLR-6654

          Show
          ASF subversion and git services added a comment - Commit 1636862 from Noble Paul in branch 'dev/trunk' [ https://svn.apache.org/r1636862 ] SOLR-6533 , SOLR-6654
          Hide
          ASF subversion and git services added a comment -

          Commit 1636863 from Noble Paul in branch 'dev/trunk'
          [ https://svn.apache.org/r1636863 ]

          SOLR-6533

          Show
          ASF subversion and git services added a comment - Commit 1636863 from Noble Paul in branch 'dev/trunk' [ https://svn.apache.org/r1636863 ] SOLR-6533
          Hide
          Noble Paul added a comment -

          This patch adds a watch to solrconfig.xml as well

          Show
          Noble Paul added a comment - This patch adds a watch to solrconfig.xml as well
          Hide
          Noble Paul added a comment -

          Added a testcase for config reload and refactored the listening

          Show
          Noble Paul added a comment - Added a testcase for config reload and refactored the listening
          Hide
          ASF subversion and git services added a comment -

          Commit 1640564 from Mark Miller in branch 'dev/trunk'
          [ https://svn.apache.org/r1640564 ]

          SOLR-6533: Fixes the formatting for the CHANGES entry ...

          Show
          ASF subversion and git services added a comment - Commit 1640564 from Mark Miller in branch 'dev/trunk' [ https://svn.apache.org/r1640564 ] SOLR-6533 : Fixes the formatting for the CHANGES entry ...
          Hide
          ASF subversion and git services added a comment -

          Commit 1640857 from Noble Paul in branch 'dev/trunk'
          [ https://svn.apache.org/r1640857 ]

          SOLR-6533 Added a testcase for config reload, hardened watching for changes

          Show
          ASF subversion and git services added a comment - Commit 1640857 from Noble Paul in branch 'dev/trunk' [ https://svn.apache.org/r1640857 ] SOLR-6533 Added a testcase for config reload, hardened watching for changes
          Hide
          Noble Paul added a comment -

          the patch for the latest commit

          Show
          Noble Paul added a comment - the patch for the latest commit
          Hide
          ASF subversion and git services added a comment -
          Show
          ASF subversion and git services added a comment - Commit 1640909 from Noble Paul in branch 'dev/trunk' [ https://svn.apache.org/r1640909 ] SOLR-6533 fixing test failures http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11645
          Hide
          ASF subversion and git services added a comment -
          Show
          ASF subversion and git services added a comment - Commit 1641020 from Noble Paul in branch 'dev/trunk' [ https://svn.apache.org/r1641020 ] SOLR-6533 fixing test failures http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4445
          Hide
          ASF subversion and git services added a comment -

          Commit 1641420 from Noble Paul in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1641420 ]

          SOLR-6533 , SOLR-6715 , SOLR-6654 Backporting all the changes from trunk

          Show
          ASF subversion and git services added a comment - Commit 1641420 from Noble Paul in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1641420 ] SOLR-6533 , SOLR-6715 , SOLR-6654 Backporting all the changes from trunk
          Hide
          Alexandre Rafalovitch added a comment -

          What happens if we set and then unset a property that overrides the one in solrconfig.xml? Do we revert to the original hardcoded one?

          Show
          Alexandre Rafalovitch added a comment - What happens if we set and then unset a property that overrides the one in solrconfig.xml? Do we revert to the original hardcoded one?
          Hide
          Noble Paul added a comment -

          What happens if we set and then unset a property that overrides the one in solrconfig.xml? Do we revert to the original hardcoded one?

          "hardcoded" ? do you mean what is specified in solrconfig.xml ? yes.

          Show
          Noble Paul added a comment - What happens if we set and then unset a property that overrides the one in solrconfig.xml? Do we revert to the original hardcoded one? "hardcoded" ? do you mean what is specified in solrconfig.xml ? yes.
          Hide
          ASF subversion and git services added a comment -

          Commit 1649486 from Noble Paul in branch 'dev/trunk'
          [ https://svn.apache.org/r1649486 ]

          SOLR-6533 , changed the node name from solrConfig to config. handle empty command list

          Show
          ASF subversion and git services added a comment - Commit 1649486 from Noble Paul in branch 'dev/trunk' [ https://svn.apache.org/r1649486 ] SOLR-6533 , changed the node name from solrConfig to config. handle empty command list
          Hide
          ASF subversion and git services added a comment -

          Commit 1649487 from Noble Paul in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1649487 ]

          SOLR-6533 , changed the node name from solrConfig to config. handle empty command list

          Show
          ASF subversion and git services added a comment - Commit 1649487 from Noble Paul in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1649487 ] SOLR-6533 , changed the node name from solrConfig to config. handle empty command list
          Hide
          Alexandre Rafalovitch added a comment -

          This (in Solr 5) does not seem to return implicit handlers when I request /config. Is there a different version of a call for that?

          I strongly believe we discussed that something needs to return all the handlers. This is needed for the tools and for troubleshooting. Especially now that "/update" is an implicit handler, yet we have an initParam section for it.

          Also, what's an example of /config/<component-name> ? I could not quickly figure out what the component-name would be in this context.

          Show
          Alexandre Rafalovitch added a comment - This (in Solr 5) does not seem to return implicit handlers when I request /config . Is there a different version of a call for that? I strongly believe we discussed that something needs to return all the handlers. This is needed for the tools and for troubleshooting. Especially now that "/update" is an implicit handler, yet we have an initParam section for it. Also, what's an example of /config/<component-name> ? I could not quickly figure out what the component-name would be in this context.
          Hide
          Noble Paul added a comment - - edited

          You are right, The /config is not aware of implicitly defined plugins. We need it to make it available

          Also, what's an example of /config/<component-name>

          component names are requestHandler, searchComponent etc. I should document that

          Show
          Noble Paul added a comment - - edited You are right, The /config is not aware of implicitly defined plugins. We need it to make it available Also, what's an example of /config/<component-name> component names are requestHandler, searchComponent etc. I should document that
          Hide
          ASF subversion and git services added a comment -

          Commit 1650000 from Noble Paul in branch 'dev/trunk'
          [ https://svn.apache.org/r1650000 ]

          SOLR-6533 implicit requesthandlers were not exposed

          Show
          ASF subversion and git services added a comment - Commit 1650000 from Noble Paul in branch 'dev/trunk' [ https://svn.apache.org/r1650000 ] SOLR-6533 implicit requesthandlers were not exposed
          Hide
          ASF subversion and git services added a comment -

          Commit 1650001 from Noble Paul in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1650001 ]

          SOLR-6533 implicit requesthandlers were not exposed

          Show
          ASF subversion and git services added a comment - Commit 1650001 from Noble Paul in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1650001 ] SOLR-6533 implicit requesthandlers were not exposed
          Hide
          Noble Paul added a comment -

          The implicit handlers are also exposed now

          Show
          Noble Paul added a comment - The implicit handlers are also exposed now
          Hide
          ASF subversion and git services added a comment -

          Commit 1658916 from Noble Paul in branch 'dev/trunk'
          [ https://svn.apache.org/r1658916 ]

          SOLR-6533: logging added and formatting fixed

          Show
          ASF subversion and git services added a comment - Commit 1658916 from Noble Paul in branch 'dev/trunk' [ https://svn.apache.org/r1658916 ] SOLR-6533 : logging added and formatting fixed
          Hide
          ASF subversion and git services added a comment -

          Commit 1658917 from Noble Paul in branch 'dev/trunk'
          [ https://svn.apache.org/r1658917 ]

          SOLR-6533: formatting fixed

          Show
          ASF subversion and git services added a comment - Commit 1658917 from Noble Paul in branch 'dev/trunk' [ https://svn.apache.org/r1658917 ] SOLR-6533 : formatting fixed
          Hide
          ASF subversion and git services added a comment -

          Commit 1658918 from Noble Paul in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1658918 ]

          SOLR-6533: logging added and formatting fixed

          Show
          ASF subversion and git services added a comment - Commit 1658918 from Noble Paul in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1658918 ] SOLR-6533 : logging added and formatting fixed
          Hide
          ASF subversion and git services added a comment -

          Commit 1658920 from Noble Paul in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1658920 ]

          SOLR-6533: logging added and formatting fixed

          Show
          ASF subversion and git services added a comment - Commit 1658920 from Noble Paul in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1658920 ] SOLR-6533 : logging added and formatting fixed
          Hide
          Anshum Gupta added a comment -

          Bulk close after 5.0 release.

          Show
          Anshum Gupta added a comment - Bulk close after 5.0 release.

            People

            • Assignee:
              Noble Paul
              Reporter:
              Noble Paul
            • Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development