Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-6498

Elasticsearch multi-field mappings do not work

    XMLWordPrintableJSON

Details

    Description

      EmbeddedElasticsearchPolicy#
      applyMapping tries to apply a multi-field mapping by adding another properties field inside a parent field mapping. In Elasticsearch you need to use fields rather than properties for a multi-field mapping to work.
       
      I've tested this with ES 7 and ES 6 (only including ES 7 example here in the issue description; for ES 6 you need to wrap everything under "mappings" in "_doc"):

      ES 7:

      PUT /some_index
      {
        "mappings": {
          "properties": {
            "some_field": {
              "type": "text",
              "properties": {
                "keyword": {
                  "type": "keyword"
                }
              }
            }
          }
        }
      }

      will lead to the following error:

      {
        "error" : {
          "root_cause" : [
            {
              "type" : "mapper_parsing_exception",
              "reason" : "Mapping definition for [some_field] has unsupported parameters:  [properties : {keyword={type=keyword}}]"
            }
          ],
          "type" : "mapper_parsing_exception",
          "reason" : "Failed to parse mapping [_doc]: Mapping definition for [some_field] has unsupported parameters:  [properties : {keyword={type=keyword}}]",
          "caused_by" : {
            "type" : "mapper_parsing_exception",
            "reason" : "Mapping definition for [some_field] has unsupported parameters:  [properties : {keyword={type=keyword}}]"
          }
        },
        "status" : 400
      } 

       
      Successful request:

      PUT /some_index
      {
        "mappings": {
          "properties": {
            "some_field": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword"
                }
              }
            }
          }
        }
      } 

      You'll encounter this error also, if you adapt the test data in ElasticsearchAdapterTest to include nested fields:

      @BeforeAll
      public static void setupInstance() throws Exception {
        // "city.keyword" is a nested field with type "keyword"
        final Map<String, String> mapping =
            ImmutableMap.of("city", "text", "city.keyword", "keyword", "state", "keyword", "pop", "long"); 

      Error:

      {...
      
      {"type":"mapper_parsing_exception","reason":"unknown parameter [properties] on mapper [city] of type [text]"}
      
      ...} 

      Looking at ElasticsearchJson#visitMappingProperties I assume this will lead to a similar issue, which I'll double-check (we probably need to distinguish between nested fields and multi-field mappings explicitly).

      This is related to https://issues.apache.org/jira/browse/CALCITE-3027 as you want to detect, if you perform a "LIKE" operator on a purely "text" mapped field, which will lead to weird semantics ("text" mapped fields are analyzed/broken up into several tokens), if you do not prevent it. Usually you have a multi-field mapping for a field containing text ("text" mapping for full text search and a "keyword" mapping you can use for aggregations, wildcard queries etc.). So this is rather important to work correctly overall and in the tests.

      Attachments

        Issue Links

          Activity

            People

              timgrein Tim Grein
              timgrein Tim Grein
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: