[CALCITE-3023] Upgrade to Elasticsearch 7.0.0

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.20.0
• Component/s: elasticsearch-adapter
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Takako Shimamoto
• Created: 25/Apr/19 02:11
• Updated: 24/Jun/19 15:02
• Resolved: 21/May/19 23:13

Description

Types are deprecated in APIs in 7.0, with breaking changes to the index creation, put mapping, get mapping, put template, get template and get field mappings APIs.
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/removal-of-types.html

In this ticket, I propose that the name of the tables corresponds to indices in Elasticsearch.

For example:

model.json

{
  "version": "1.0",
  "defaultSchema": "elasticsearch",
  "schemas": [
    {
      "type": "custom",
      "name": "elasticsearch",
      "factory": "org.apache.calcite.adapter.elasticsearch.ElasticsearchSchemaFactory",
      "operand": {
        "coordinates": "{'127.0.0.1': 9200}"
      }
    }
  ]
}

indices in Elasticsearch

root@50153d5029b6:/calcite# curl localhost:9200
{
  "name" : "50153d5029b6",
  "cluster_name" : "predictionio",
  "cluster_uuid" : "7drQAZjASgO5SFuuS7zXBA",
  "version" : {
    "number" : "7.0.0",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "b7e28a7",
    "build_date" : "2019-04-05T22:55:32.697037Z",
    "build_snapshot" : false,
    "lucene_version" : "8.0.0",
    "minimum_wire_compatibility_version" : "6.7.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}
root@50153d5029b6:/calcite# curl -XGET 'http://localhost:9200/_cat/indices?v'
health status index   uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   usa     tHUdsJZLRRS57knDyfD4Bg   1   1          2            0     10.1kb         10.1kb
yellow open   twitter AtiirLwyQXKCFNFrnYCMVQ   1   1          1            0      4.5kb          4.5kb

We hope it displays like this:

root@50153d5029b6:/calcite# ./sqlline 
sqlline version 1.7.0
sqlline> !connect jdbc:calcite:model=model.json admin admin
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/root/.m2/repository/org/slf4j/slf4j-log4j12/1.7.25/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/root/.m2/repository/ch/qos/logback/logback-classic/1.1.3/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Transaction isolation level TRANSACTION_REPEATABLE_READ is not supported. Default (TRANSACTION_NONE) will be used instead.
0: jdbc:calcite:model=model.json> SELECT * from "usa";
+------------------------------------------------------------------------------------+
|                                        _MAP                                        |
+------------------------------------------------------------------------------------+
| {pop=13367, loc=[-72.505565, 42.067203], city=EAST LONGMEADOW, id=01028, state=MA} |
| {pop=1350, loc=[-72.703403, 42.354292], city=LEEDS, id=01053, state=MA}            |
+------------------------------------------------------------------------------------+
2 rows selected (2.472 seconds)
0: jdbc:calcite:model=model.json> SELECT * from "twitter";
+--------------------------------------------------------------------------------+
|                                      _MAP                                      |
+--------------------------------------------------------------------------------+
| {user=kimchy, post_date=2009-11-15T14:12:12, message=trying out Elasticsearch} |
+--------------------------------------------------------------------------------+
1 row selected (0.091 seconds)
0: jdbc:calcite:model=model.json> SELECT count(*) from "usa";
+--------+
| EXPR$0 |
+--------+
| 2      |
+--------+
1 row selected (0.404 seconds)
0: jdbc:calcite:model=model.json> SELECT count(*) from "twitter";
+--------+
| EXPR$0 |
+--------+
| 1      |
+--------+
1 row selected (0.136 seconds)
0: jdbc:calcite:model=model.json> SELECT _MAP['pop'] as "pop", _MAP['city'] as "city" from "usa";
+-------+-----------------+
|  pop  |      city       |
+-------+-----------------+
| 13367 | EAST LONGMEADOW |
| 1350  | LEEDS           |
+-------+-----------------+
2 rows selected (0.121 seconds)
0: jdbc:calcite:model=model.json> SELECT _MAP['pop'] as "pop", _MAP['city'] as "city" from "usa" where _MAP['pop'] > 10000;
+-------+-----------------+
|  pop  |      city       |
+-------+-----------------+
| 13367 | EAST LONGMEADOW |
+-------+-----------------+
1 row selected (0.306 seconds)
0: jdbc:calcite:model=model.json> SELECT _MAP['user'] as "user", _MAP['message'] as "message" from "twitter";
+--------+--------------------------+
|  user  |         message          |
+--------+--------------------------+
| kimchy | trying out Elasticsearch |
+--------+--------------------------+
1 row selected (0.133 seconds)
0: jdbc:calcite:model=model.json> 

Issue Links

• links to: GitHub Pull Request #1180