Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
Description
EmbeddedElasticsearchPolicy#
applyMapping tries to apply a multi-field mapping by adding another properties field inside a parent field mapping. In Elasticsearch you need to use fields rather than properties for a multi-field mapping to work.
I've tested this with ES 7 and ES 6 (only including ES 7 example here in the issue description; for ES 6 you need to wrap everything under "mappings" in "_doc"):
ES 7:
PUT /some_index { "mappings": { "properties": { "some_field": { "type": "text", "properties": { "keyword": { "type": "keyword" } } } } } }
will lead to the following error:
{ "error" : { "root_cause" : [ { "type" : "mapper_parsing_exception", "reason" : "Mapping definition for [some_field] has unsupported parameters: [properties : {keyword={type=keyword}}]" } ], "type" : "mapper_parsing_exception", "reason" : "Failed to parse mapping [_doc]: Mapping definition for [some_field] has unsupported parameters: [properties : {keyword={type=keyword}}]", "caused_by" : { "type" : "mapper_parsing_exception", "reason" : "Mapping definition for [some_field] has unsupported parameters: [properties : {keyword={type=keyword}}]" } }, "status" : 400 }
Successful request:
PUT /some_index { "mappings": { "properties": { "some_field": { "type": "text", "fields": { "keyword": { "type": "keyword" } } } } } }
You'll encounter this error also, if you adapt the test data in ElasticsearchAdapterTest to include nested fields:
@BeforeAll public static void setupInstance() throws Exception { // "city.keyword" is a nested field with type "keyword" final Map<String, String> mapping = ImmutableMap.of("city", "text", "city.keyword", "keyword", "state", "keyword", "pop", "long");
Error:
{... {"type":"mapper_parsing_exception","reason":"unknown parameter [properties] on mapper [city] of type [text]"} ...}
Looking at ElasticsearchJson#visitMappingProperties I assume this will lead to a similar issue, which I'll double-check (we probably need to distinguish between nested fields and multi-field mappings explicitly).
This is related to https://issues.apache.org/jira/browse/CALCITE-3027 as you want to detect, if you perform a "LIKE" operator on a purely "text" mapped field, which will lead to weird semantics ("text" mapped fields are analyzed/broken up into several tokens), if you do not prevent it. Usually you have a multi-field mapping for a field containing text ("text" mapping for full text search and a "keyword" mapping you can use for aggregations, wildcard queries etc.). So this is rather important to work correctly overall and in the tests.
Attachments
Issue Links
- blocks
-
CALCITE-6582 Release Calcite 1.38.0
- Closed
- links to