Uploaded image for project: 'Apache Trafodion'
  1. Apache Trafodion
  2. TRAFODION-2584

Add support to register hive objects in trafodion metadata

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Many trafodion operations on hive objects require an objectUID (a unique
      identifier) to be associated with them. This allows them to be added to
      trafodion metadata and then referred to later.

      Operations that need an identifier are privileges grant/revoke,
      updating statistics, hive view usage and external tables.

      Currently, to associate an objectUID with a hive table, trafodion creates
      an implicit external table on it. That generates a UID for the table
      which is then associated with the underlying object.
      That approach is not lightweight in that the implicit external table
      creation impact multiple traf metadata tables as it goes through all the
      steps needed for table creation.
      It doesn't support hive views as traf doesnt allow creation of external
      tables on a hive view.

      Also, If an implicit external table has been created by a traf operation,
      for example update statistics, and then the user wants to create an
      explicit external table, they need to drop the implicit table before
      creating the new external table. It means that all stats become
      obsolete as they are associated with the older implicit external
      table objectUID.

      This enhancement proposes a way to register hive objects(tables, views)
      in trafodion metadata OBJECTS without creating an external table.

      That has the following advantages:
      – registering objects is a lightweight operation without creating an
      external table
      – it impact only OBJECTS metadata and doesnt affect other objects
      (like COLUMNS, KEYS, etc) which will be for an external table
      – it can be used for hive tables and views
      – since there is no implicit external table, this allows creation of user
      external tables without having to drop the implicit table and thus
      invalidating the older objectUID. User external table create/drop
      become independent of internal operations that require an objectUID.

        Attachments

          Activity

            People

            • Assignee:
              anoopsharma Anoop Sharma
              Reporter:
              anoopsharma Anoop Sharma
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: