Solr
  1. Solr
  2. SOLR-3967

Mapping error: langid.enforceSchema option checks source field instead of target field

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 4.0
    • Fix Version/s: 4.2, 6.0
    • Component/s: contrib - LangId
    • Labels:
      None

      Description

      I use LangDetect update processor with a document that has "body" field. LangDetect should map this field to "body_pl", "body_en" or "body_nolang". My schema defines fields with language suffixes, but not "body" field. When the processor runs, I get error:

      Unsuccessful field name mapping to body_nolang, field does not exist, skipping mapping.

      I looked up source code and it seems there's an error in org.apache.solr.update.processor.LanguageIdentifierUpdateProcessor.process(SolrInputDocument):

                String mappedOutputField = getMappedField(fieldName, fieldLang);
                if(enforceSchema && schema.getFieldOrNull(fieldName) == null) {
                  log.warn("Unsuccessful field name mapping to {}, field does not exist, skipping mapping.", mappedOutputField, fieldName);
                  mappedOutputField = fieldName;
                }
      

      I think it should check for schema.getFieldOrNull(mappedOutputField) instead.

      1. SOLR-3967.patch
        5 kB
        Jan Høydahl

        Activity

        Hide
        Alexey Kudinov added a comment -

        Apparently it should check the mapped field

        Show
        Alexey Kudinov added a comment - Apparently it should check the mapped field
        Hide
        Jan Høydahl added a comment -

        Patch for trunk, including positive/negative test for enforceSchema option

        Show
        Jan Høydahl added a comment - Patch for trunk, including positive/negative test for enforceSchema option
        Hide
        Jan Høydahl added a comment -

        The patch moves the enforce logic to the getMappedField() method.

        I also fixed an issue where langid would fallback to original field instead of aborting in case mapped field does not exist. Will commit shortly if no objections.

        Show
        Jan Høydahl added a comment - The patch moves the enforce logic to the getMappedField() method. I also fixed an issue where langid would fallback to original field instead of aborting in case mapped field does not exist. Will commit shortly if no objections.
        Hide
        Commit Tag Bot added a comment -

        [trunk commit] Jan Høydahl
        http://svn.apache.org/viewvc?view=revision&revision=1440226

        SOLR-3967: langid.enforceSchema option checks source field instead of target field

        Show
        Commit Tag Bot added a comment - [trunk commit] Jan Høydahl http://svn.apache.org/viewvc?view=revision&revision=1440226 SOLR-3967 : langid.enforceSchema option checks source field instead of target field
        Hide
        Commit Tag Bot added a comment -

        [branch_4x commit] Jan Høydahl
        http://svn.apache.org/viewvc?view=revision&revision=1440319

        SOLR-3967: langid.enforceSchema option checks source field instead of target field (merge from trunk)

        Show
        Commit Tag Bot added a comment - [branch_4x commit] Jan Høydahl http://svn.apache.org/viewvc?view=revision&revision=1440319 SOLR-3967 : langid.enforceSchema option checks source field instead of target field (merge from trunk)
        Hide
        Jan Høydahl added a comment -

        This should now be fixed. Perhaps you Mateusz Matela would care to check out the code and verify if you're happy?

        Show
        Jan Høydahl added a comment - This should now be fixed. Perhaps you Mateusz Matela would care to check out the code and verify if you're happy?
        Hide
        Mateusz Matela added a comment -

        Yes, the fix looks good. Thanks!

        Show
        Mateusz Matela added a comment - Yes, the fix looks good. Thanks!

          People

          • Assignee:
            Jan Høydahl
            Reporter:
            Mateusz Matela
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development