Details
-
New Feature
-
Status: Closed
-
Major
-
Resolution: Fixed
-
2.0-incubating
-
None
Description
Trafodion supports selecting, loading from, listing, and describing native HBase tables. HBase tables are identified by specifying a special catalog called HBASE and schemas called "ROW" and "CELL".
Example to select from a native HBase table:
select left(row_id, 10) as row_id, left(column_display(column_details, ('teams:team_number', 'games:visitor_team', 'games:game_time')), 100) as cols from hbase."ROW"."baseball";
Trafodion interprets the special HBASE catalog be a native HBase table. During preparation of an SQL statement referencing a native HBase table, Trafodion contacts HBase and obtains a description of the table. It then creates an internal description (NATable) of the HBase table. The NATable definition is used by the compiler and code generation process to prepare the plan. Trafodion does not store any details in Trafodion metadata.
Several Trafodion commands today, would work more effectively if we allow native HBase tables to be partially described in Trafodion. That is, store their definitions in Trafodion metadata. This JIRA describes a proposal to allow native HBase tables to be registered in Trafodion metadata by specifying the “CREATE EXTERNAL <table> TABLE …” syntax.
Proposal
Allow native HBase tables to be registered in Trafodion metadata through the EXTERNAL TABLE create option.
CREATE EXTERNAL TABLE [IF NOT EXISTS] table FOR hbase-source-table;
DROP EXTERNAL TABLE [IF EXISTS] table FOR hbase-source_table
hbase-source-table - native HBase table to be registered in the Trafodion metadata. The hbase-source-table has to exist.
table - table stored in Trafodion metadata. Initially, the table name should be the same as the hbase-source-table name.
The default catalog name for HBase tables are HBASE (defined in ComSmallDefs as HBASE_SYSTEM_CATALOG).
To change the description, the external table needs to be dropped then recreated. ALTER EXTERNAL TABLE is not supported.
The following command’s behavior changes for external tables:
• UPDATE STATISTICS – an external table can be used to gather statistics for HIVE tables
• SHOWDDL – will now be allowed on external tables
• GRANT and REVOKE – privileges can be specified for external tables
• SELECT and LOAD – will be allowed on external tables
This is a sister project to JIRA TRAFODION-19 and implemented in much the same way.
