Uploaded image for project: 'Apache Trafodion'
  1. Apache Trafodion
  2. TRAFODION-1483

External tables for Native HBase tables

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0-incubating
    • Fix Version/s: 2.0-incubating
    • Component/s: sql-general
    • Labels:
      None

      Description

      Trafodion supports selecting, loading from, listing, and describing native HBase tables. HBase tables are identified by specifying a special catalog called HBASE and schemas called "ROW" and "CELL".
      Example to select from a native HBase table:

      select left(row_id, 10) as row_id, left(column_display(column_details, ('teams:team_number', 'games:visitor_team', 'games:game_time')), 100) as cols from hbase."ROW"."baseball";

      Trafodion interprets the special HBASE catalog be a native HBase table. During preparation of an SQL statement referencing a native HBase table, Trafodion contacts HBase and obtains a description of the table. It then creates an internal description (NATable) of the HBase table. The NATable definition is used by the compiler and code generation process to prepare the plan. Trafodion does not store any details in Trafodion metadata.

      Several Trafodion commands today, would work more effectively if we allow native HBase tables to be partially described in Trafodion. That is, store their definitions in Trafodion metadata. This JIRA describes a proposal to allow native HBase tables to be registered in Trafodion metadata by specifying the “CREATE EXTERNAL <table> TABLE …” syntax.

      Proposal

      Allow native HBase tables to be registered in Trafodion metadata through the EXTERNAL TABLE create option.
      CREATE EXTERNAL TABLE [IF NOT EXISTS] table FOR hbase-source-table;
      DROP EXTERNAL TABLE [IF EXISTS] table FOR hbase-source_table

      hbase-source-table - native HBase table to be registered in the Trafodion metadata. The hbase-source-table has to exist.

      table - table stored in Trafodion metadata. Initially, the table name should be the same as the hbase-source-table name.

      The default catalog name for HBase tables are HBASE (defined in ComSmallDefs as HBASE_SYSTEM_CATALOG).
      To change the description, the external table needs to be dropped then recreated. ALTER EXTERNAL TABLE is not supported.
      The following command’s behavior changes for external tables:
      • UPDATE STATISTICS – an external table can be used to gather statistics for HIVE tables
      • SHOWDDL – will now be allowed on external tables
      • GRANT and REVOKE – privileges can be specified for external tables
      • SELECT and LOAD – will be allowed on external tables

      This is a sister project to JIRA TRAFODION-19 and implemented in much the same way.

        Attachments

          Activity

            People

            • Assignee:
              rmarton Roberta Marton
              Reporter:
              rmarton Roberta Marton
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Due:
                Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 168h
                168h
                Remaining:
                Remaining Estimate - 168h
                168h
                Logged:
                Time Spent - Not Specified
                Not Specified