Solr
  1. Solr
  2. SOLR-3200

When using SignatureUpdateProcessor with "all fields" configuration, it will assume only the fields present on the very first document only, ignoring any optional fields in subsequent documents in the signature generation.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.4, 3.1, 3.2, 3.3, 3.4, 3.5, 4.0-ALPHA
    • Fix Version/s: 3.6, 4.0-ALPHA
    • Component/s: update
    • Labels:
      None

      Description

      This can result in non-duplicate documents being left out of the index. A solution would be that the fields to be used in the signature generation are recalculated with every document inserted.

      1. SOLR-3200.patch
        4 kB
        Spyros Kapnissis

        Issue Links

          Activity

          Hide
          Spyros Kapnissis added a comment -

          You're welcome

          Another thing that is not so intuitive here is when using the "all fields" configuration on a schema that has a unique key defined. This makes the whole process redundant as deduplication is already covered by the schema's unique key.

          So maybe it would be safe to assume that this configuration always means "all fields - excluding the unique key" by adding a runtime check to exclude this field from the signature calculation?

          Show
          Spyros Kapnissis added a comment - You're welcome Another thing that is not so intuitive here is when using the "all fields" configuration on a schema that has a unique key defined. This makes the whole process redundant as deduplication is already covered by the schema's unique key. So maybe it would be safe to assume that this configuration always means "all fields - excluding the unique key" by adding a runtime check to exclude this field from the signature calculation?
          Hide
          Hoss Man added a comment -

          Committed revision 1308625. - 3x

          Thank you so much for your patch(es) Spyros

          Show
          Hoss Man added a comment - Committed revision 1308625. - 3x Thank you so much for your patch(es) Spyros
          Hide
          Hoss Man added a comment -

          Committed revision 1308604. - trunk

          still testing backport to 3x

          Show
          Hoss Man added a comment - Committed revision 1308604. - trunk still testing backport to 3x
          Hide
          Hoss Man added a comment -

          Grr... another one that looks really bad.

          Patch and tests look good to me (but still doing more testing) so I think we should commit for 3.6

          (Same upgrade instructions as SOLR-3226 would cover this)

          Show
          Hoss Man added a comment - Grr... another one that looks really bad. Patch and tests look good to me (but still doing more testing) so I think we should commit for 3.6 (Same upgrade instructions as SOLR-3226 would cover this)
          Hide
          Spyros Kapnissis added a comment -

          I am attaching a patch with the test case and a proposed solution.

          Show
          Spyros Kapnissis added a comment - I am attaching a patch with the test case and a proposed solution.

            People

            • Assignee:
              Hoss Man
              Reporter:
              Spyros Kapnissis
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development