Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-9411

Better validation of fields and dynamic fields for Schema API

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 6.3, 7.0
    • Component/s: Schema and Analysis
    • Security Level: Public (Default Security Level. Issues are Public)
    • Labels:
      None

      Description

      Schema REST API needs better validation before doing changes.

      • It should not be allowed to delete uniqueKey (also handled in SOLR-9349)
      • When adding a dynamic field the API should test that it begins or ends with *. Today the change succeeds, but you get errors later

      These are two known cases. We should harden validation across the board for all known schema requirements.

      1. SOLR-9411-part2.patch
        4 kB
        Steve Rowe
      2. SOLR-9411-part2.patch
        5 kB
        Steve Rowe
      3. SOLR-9411.patch
        3 kB
        Jan Høydahl
      4. SOLR-9411.patch
        6 kB
        Jan Høydahl

        Issue Links

          Activity

          Hide
          janhoy Jan Høydahl added a comment -

          This patch fixes a bug in add-dynamic-field, where the field got created using SchemaField.create() instead of managedIndexSchema.newDynamicField()

          Show
          janhoy Jan Høydahl added a comment - This patch fixes a bug in add-dynamic-field , where the field got created using SchemaField.create() instead of managedIndexSchema.newDynamicField()
          Hide
          janhoy Jan Høydahl added a comment -

          Updated patch.

          Fixes SchemaTest which attempted to create dynamic fields with required=true and defaultValue. This succeeded earlier due to this same bug

          Show
          janhoy Jan Høydahl added a comment - Updated patch. Fixes SchemaTest which attempted to create dynamic fields with required=true and defaultValue. This succeeded earlier due to this same bug
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 8046fe29e55e46df4d5093460825f64dabd4b7ba in lucene-solr's branch refs/heads/master from Jan Høydahl
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=8046fe2 ]

          SOLR-9411: Better validation of dynamic field for Schema API

          Show
          jira-bot ASF subversion and git services added a comment - Commit 8046fe29e55e46df4d5093460825f64dabd4b7ba in lucene-solr's branch refs/heads/master from Jan Høydahl [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=8046fe2 ] SOLR-9411 : Better validation of dynamic field for Schema API
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 53fcc7563c1010e34a8f0889968711816bcb89ff in lucene-solr's branch refs/heads/branch_6x from Jan Høydahl
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=53fcc75 ]

          SOLR-9411: Better validation of dynamic field for Schema API

          (cherry picked from commit 8046fe2)

          Show
          jira-bot ASF subversion and git services added a comment - Commit 53fcc7563c1010e34a8f0889968711816bcb89ff in lucene-solr's branch refs/heads/branch_6x from Jan Høydahl [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=53fcc75 ] SOLR-9411 : Better validation of dynamic field for Schema API (cherry picked from commit 8046fe2)
          Hide
          steve_rowe Steve Rowe added a comment -

          +1 to your changes Jan Høydahl.

          This add-on patch:

          • Switches the "add-field" operation to use IndexSchema.newField() instead of SchemaField.create(), to take advantage of the more extensive validation there.
          • Adds a test to TestBulkSchemaAPI that attempts to add invalid fields using "add-field".
          • Removes the fieldtype existence check from the "add-field" and "add-dynamic-field" operations, since IndexSchema.new(Dynamic)Field() already handles this validation.

          Running tests now.

          Show
          steve_rowe Steve Rowe added a comment - +1 to your changes Jan Høydahl . This add-on patch: Switches the "add-field" operation to use IndexSchema.newField() instead of SchemaField.create(), to take advantage of the more extensive validation there. Adds a test to TestBulkSchemaAPI that attempts to add invalid fields using "add-field". Removes the fieldtype existence check from the "add-field" and "add-dynamic-field" operations, since IndexSchema.new(Dynamic)Field() already handles this validation. Running tests now.
          Hide
          janhoy Jan Høydahl added a comment -

          Good catch. Feel free to piggy-back on this issue and adjust title and CHANGES entry accordingly.

          Show
          janhoy Jan Høydahl added a comment - Good catch. Feel free to piggy-back on this issue and adjust title and CHANGES entry accordingly.
          Hide
          steve_rowe Steve Rowe added a comment -

          Reopening to commit my patch.

          Show
          steve_rowe Steve Rowe added a comment - Reopening to commit my patch.
          Hide
          steve_rowe Steve Rowe added a comment -

          Slight modification to an existing test in TestBulkSchemaAPI, since the error message for invalid "add-field" changed with the switch to IndexSchema.newField(). Also modified CHANGES.txt entry.

          Tests and precommit pass with this patch. Committing shortly.

          Show
          steve_rowe Steve Rowe added a comment - Slight modification to an existing test in TestBulkSchemaAPI, since the error message for invalid "add-field" changed with the switch to IndexSchema.newField(). Also modified CHANGES.txt entry. Tests and precommit pass with this patch. Committing shortly.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 49c5a749c3e95d33ca338331a270633503fe8ff7 in lucene-solr's branch refs/heads/branch_6x from Steve Rowe
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=49c5a74 ]

          SOLR-9411: Better validation for Schema API add-field

          Show
          jira-bot ASF subversion and git services added a comment - Commit 49c5a749c3e95d33ca338331a270633503fe8ff7 in lucene-solr's branch refs/heads/branch_6x from Steve Rowe [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=49c5a74 ] SOLR-9411 : Better validation for Schema API add-field
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 9f35a6b829aff968d506239b0945266fc7866f65 in lucene-solr's branch refs/heads/master from Steve Rowe
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9f35a6b ]

          SOLR-9411: Better validation for Schema API add-field

          Show
          jira-bot ASF subversion and git services added a comment - Commit 9f35a6b829aff968d506239b0945266fc7866f65 in lucene-solr's branch refs/heads/master from Steve Rowe [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9f35a6b ] SOLR-9411 : Better validation for Schema API add-field
          Hide
          shalinmangar Shalin Shekhar Mangar added a comment -

          Closing after 6.3.0 release.

          Show
          shalinmangar Shalin Shekhar Mangar added a comment - Closing after 6.3.0 release.

            People

            • Assignee:
              steve_rowe Steve Rowe
              Reporter:
              janhoy Jan Høydahl
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development