Details
Description
Currently, Tajo uses HDFS as a primary storage. But, as a data warehouse system, Tajo should easily support various data sources.
For this, I propose a generic storage handler interface that provides common storage methods as follows:
- splitting input data
- locality
- accessing catalog (if providing)
- creating a table
- removing a table
- adding default table properties and validating properties
- committing, rollback, and clean up output tables
- getting table physical information like table volumes and others
- managing connection pool for connection-based storages
- adding storage-specified rewrite rules
- adding hooks for query phases
- physical properties like instant random access, indexible, read throughput, and write throughput
Attachments
Attachments
Issue Links
- incorporates
-
TAJO-1163 TableDesc should use URI instead of Path.
-
- Resolved
-
- is related to
-
TAJO-1127 Implements HBaseStorageManager
-
- Resolved
-
-
TAJO-1123 Use Fragment instead of FileFragment.
-
- Resolved
-
-
TAJO-1879 Add enabled and description fields to tablespace entry in storage-site.json
-
- Open
-
- is required by
-
TAJO-367 Separate the locality information from Fragment
-
- Resolved
-
-
TAJO-1595 Pluggable Storage Handler
-
- Resolved
-
1.
|
Refactor StorageManager |
|
Resolved | Hyunsik Choi |
2.
|
Rename StorageManager to Tablespace |
|
Resolved | Hyunsik Choi |
3.
|
Configuration format proposal for generic storage support |
|
Resolved | Hyunsik Choi |
4.
|
Implement TablespaceManager to load Tablespaces |
|
Resolved | Hyunsik Choi |
5.
|
Storage configuration reader |
|
Resolved | Unassigned |
6.
|
Automatic metadata registeration from underlying storages |
|
Resolved | Hyunsik Choi |
7.
|
Add 'INSERT INTO VALUES' statement for OLTP-like storages |
|
Resolved | Hyunsik Choi |
8.
|
Add 'UPDATE' statement for OLTP-like storages |
|
Open | Unassigned |
9.
|
Add 'Upsert' statement for OLTP-like storages |
|
Open | Unassigned |
10.
|
Index support of underlying storages |
|
Open | Unassigned |
11.
|
Filter push down to underlying storages |
|
Resolved | Hyunsik Choi |
12.
|
Implement formats loading and registration |
|
Open | Unassigned |
13.
|
Change the variable name storeType to dataFormat |
|
Resolved | Hyunsik Choi |
14.
|
CREATE EXTERNAL TABLE should allows just a path. |
|
Resolved | Hyunsik Choi |