[CALCITE-210] Hazelcast schema

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Labels: github-import
• Assignee: Unassigned
• Reporter: GitHub Import
• Created: 18/Jun/14 08:37
• Updated: 18/Jun/14 08:37

Description

@deinspanjer proposed an implementation of the Schema SPI on top of Hazelcast. This will allow metadata to automatically propagate among nodes in a cluster of Optiq instances.

Similar schemas could be created on top of other data grids (GridGain, JBoss Infinispan).

The core of Hazelcast is a distributed map that automatically synchronizes among nodes of a cluster. We propose an implementation of `SchemaFactory` that creates a schema that uses such a map. The nodes in the cluster would be able to add sub-schemas, tables and functions (including views and materializations) to a schema and other nodes would see them.

A schema definition is fairly easily serializable, so it should just work.

One problem might be to ensure that the contents of tables (in particular large in-memory tables like `ArrayTable`) are not moved between nodes. (It would be great if the data grid makes these accessible, but we need to control the propagation. If a table is large, it is better to send the query to the data, rather than pull the data to the query.)

The following example creates a schema `hazel` with a sub-schema `foodmart` and a view `female_emps`.

```json
{
version: '1.0',
defaultSchema: 'foodmart',
schemas: [
{
type: 'custom',
name: 'hazel',
factory: 'net.hydromatic.optiq.impl.hazelcast.HazelcastSchemaFactory',
operand:

{ configUri: '/my-hazelcast.xml' }

,
schemas: [

{ type: 'jdbc', name: 'foodmart', jdbcUrl: 'jdbc:mysql://db.local/foodmart', jdbcUser: 'foodmart', jdbcPassword: 'foodmart' }

],
tables: [

{ type: 'view' name: 'female_emps', sql: 'select * from "emps" where "gender" = \'F\'' }

]
}
]
}
```

Node1 could add a new view:

```java
Connection connection;
SchemaPlus rootSchema = ((OptiqConnection) connection).getRootSchema();
SchemaPlus hazelSchema = rootSchema.getSubSchema("hazel");
hazelSchema.add("male_emps",
ViewTable.viewFunction(hazelSchema,
"select * from \"emps\" where \"gender\" = 'M'", null);
```

and Node2 would see it and be able to use it in its queries.

---------------- Imported from GitHub ----------------
Url: https://github.com/julianhyde/optiq/issues/210
Created by: julianhyde
Labels: enhancement,
Created at: Tue Mar 25 22:45:22 CET 2014
State: open