Solr
  1. Solr
  2. SOLR-7967

AddSchemaFieldsUpdateProcessorFactory does not check if the ConfigSet is immutable

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 5.3, 6.0
    • Fix Version/s: 5.4, 6.0
    • Component/s: SolrCloud
    • Labels:
      None

      Description

      SOLR-7742 introduced Immutable ConfigSets. There are checks added to SolrConfigHandler and SchemaHandler so that if a user tries to modify the SolrConfig or the Schema via either of these interfaces an error is returned if the ConfigSet is defined to be immutable.

      Updates to the schema made via the AddSchemaFieldsUpdateProcessorFactory are not checked in this way. I'm not certain this should be considered a bug. A ConfigSet is defined by {SolrConfig, Schema, ConfigSetProperties}. On one hand, you can argue that you are modifying the Schema, which is part of the ConfigSet, so the immutable check should apply. On the other hand, the SolrConfig (which defines the AddSchema...Factory} defines that it wants the Config to be updated, so if you view the ConfigSet in totality you could argue nothing is really changing. I'd slightly lean towards adding the check, but could go either way.

      Other opinions?

      1. SOLR-7967.patch
        4 kB
        Gregory Chanan
      2. SOLR-7967.patch
        4 kB
        Gregory Chanan

        Issue Links

          Activity

          Hide
          Hoss Man added a comment -

          ...so if you view the ConfigSet in totality you could argue nothing is really changing. I'd slightly lean towards adding the check, but could go either way.

          I'm not all that familiar with how Immutable ConfigSets work internally, or what the "designed" usecase / usage patterns are, but i think the right questions to ask are:

          • what might users reasonably want to do to in conjunction with (Immutable) ConfigSets and a data driven schema?
          • what, logically, should be the default behavior of (Immutable) ConfigSets and a data driven schema?

          The answer(s) to the first question should all be possible even if they require special configuration – and or: if some conbinations of options XYZ give you errors telling you that the combination is invalid and needs to be changed to either XQZ or XYP to make sense and be usable.

          The answer to the second question should be the default behavior, w/o any special config.

          Show
          Hoss Man added a comment - ...so if you view the ConfigSet in totality you could argue nothing is really changing. I'd slightly lean towards adding the check, but could go either way. I'm not all that familiar with how Immutable ConfigSets work internally, or what the "designed" usecase / usage patterns are, but i think the right questions to ask are: what might users reasonably want to do to in conjunction with (Immutable) ConfigSets and a data driven schema? what, logically, should be the default behavior of (Immutable) ConfigSets and a data driven schema? The answer(s) to the first question should all be possible even if they require special configuration – and or: if some conbinations of options XYZ give you errors telling you that the combination is invalid and needs to be changed to either XQZ or XYP to make sense and be usable. The answer to the second question should be the default behavior, w/o any special config.
          Hide
          Gregory Chanan added a comment -

          Thanks for the discussion Hoss Man, those questions were very helpful.

          At a high level, the use case is to provide an immutable configuration as a "template" for others to build their own configuration on top of. This could be generic templates e.g. in a distribution or as a company/use case specific template. In SolrCloud mode, this allows an administrator to provide these templates and in combination with a "ConfigSet" API (SOLR-7789), a collection administrator can create their own configuration without writing directly to ZooKeeper (which is problematic from a security perspective).

          Given that use case, I believe the answers to your questions are:

          what, logically, should be the default behavior of (Immutable) ConfigSets and a data driven schema?

          Attempts to modify an immutable ConfigSet via a data driven schema should result in an error, since it is an attempt to modify an immutable / company sanctioned template.

          what might users reasonably want to do to in conjunction with (Immutable) ConfigSets and a data driven schema?

          Users may want to base their configuration on a template and use a data driven schema to update their own configuration. They can do this by creating their own non-immutable ConfigSet using the API (SOLR-7789) and using the data driven schema based on that template.

          Show
          Gregory Chanan added a comment - Thanks for the discussion Hoss Man, those questions were very helpful. At a high level, the use case is to provide an immutable configuration as a "template" for others to build their own configuration on top of. This could be generic templates e.g. in a distribution or as a company/use case specific template. In SolrCloud mode, this allows an administrator to provide these templates and in combination with a "ConfigSet" API ( SOLR-7789 ), a collection administrator can create their own configuration without writing directly to ZooKeeper (which is problematic from a security perspective). Given that use case, I believe the answers to your questions are: what, logically, should be the default behavior of (Immutable) ConfigSets and a data driven schema? Attempts to modify an immutable ConfigSet via a data driven schema should result in an error, since it is an attempt to modify an immutable / company sanctioned template. what might users reasonably want to do to in conjunction with (Immutable) ConfigSets and a data driven schema? Users may want to base their configuration on a template and use a data driven schema to update their own configuration. They can do this by creating their own non-immutable ConfigSet using the API ( SOLR-7789 ) and using the data driven schema based on that template.
          Hide
          Gregory Chanan added a comment -

          Here's a patch. It only does the immutable ConfigSet check if it's planning to actually add a new field. Interestingly, that differs from the existing mutable schema check, which checks before looking at the existing fields. So, if you had a not-mutable schema and an AddSchemaFieldsUpdateProcessorFactory you wouldn't be able to index any data, even data with all existing fields. That's probably a pre-existing bug, but seems really minor.

          Show
          Gregory Chanan added a comment - Here's a patch. It only does the immutable ConfigSet check if it's planning to actually add a new field. Interestingly, that differs from the existing mutable schema check, which checks before looking at the existing fields. So, if you had a not-mutable schema and an AddSchemaFieldsUpdateProcessorFactory you wouldn't be able to index any data, even data with all existing fields. That's probably a pre-existing bug, but seems really minor.
          Hide
          Gregory Chanan added a comment -

          SOLR-8134 for the mutable schema bug.

          Show
          Gregory Chanan added a comment - SOLR-8134 for the mutable schema bug.
          Hide
          Gregory Chanan added a comment -

          Patch with some extra null checking to get the more invasive tests to pass.

          Show
          Gregory Chanan added a comment - Patch with some extra null checking to get the more invasive tests to pass.
          Hide
          ASF subversion and git services added a comment -

          Commit 1707423 from gchanan@apache.org in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1707423 ]

          SOLR-7967: AddSchemaFieldsUpdateProcessorFactory does not check if the ConfigSet is immutable

          Show
          ASF subversion and git services added a comment - Commit 1707423 from gchanan@apache.org in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1707423 ] SOLR-7967 : AddSchemaFieldsUpdateProcessorFactory does not check if the ConfigSet is immutable
          Hide
          ASF subversion and git services added a comment -

          Commit 1707424 from gchanan@apache.org in branch 'dev/trunk'
          [ https://svn.apache.org/r1707424 ]

          SOLR-7967: AddSchemaFieldsUpdateProcessorFactory does not check if the ConfigSet is immutable

          Show
          ASF subversion and git services added a comment - Commit 1707424 from gchanan@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1707424 ] SOLR-7967 : AddSchemaFieldsUpdateProcessorFactory does not check if the ConfigSet is immutable

            People

            • Assignee:
              Gregory Chanan
              Reporter:
              Gregory Chanan
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development