Uploaded image for project: 'Phoenix'
  1. Phoenix
  2. PHOENIX-6720

CREATE TABLE can't recreate column encoded tables that had columns dropped

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 5.2.0, 5.1.3
    • 5.2.0, 5.1.4
    • core
    • None
    • Hide
      PHOENIX now supports the COLUMN_QUALIFIER_COUNTER table option and the ENCODED_QUALIFIER qualifier keyword for columns to set the "QUALIFIER_COUNTER" and "COLUMN_QUALIFIER" columns respectively in SYSTEM.CATALOG for the table.

      The CREATE TABLE statement and SchemaTool will now use the above features when generating the DDL for column encoded tables with discontinuous qualifiers.

      This enables re-generating the Phoenix metadata for tables that have discontinous encoded column qualifers, so that the re-generated Phoenix tables will work on copied or replicated HBase data tables from the source database.

      For such tables, the generated CREATE TABLE statements will only be executable for Phoenix versions having this patch. However, systems that do not have this fix cannot correctly re-generate the table metadata. On older systems the only way to achieve HBase data table level compatibility is to manually edit SYSTEM.CATALOG.
      Show
      PHOENIX now supports the COLUMN_QUALIFIER_COUNTER table option and the ENCODED_QUALIFIER qualifier keyword for columns to set the "QUALIFIER_COUNTER" and "COLUMN_QUALIFIER" columns respectively in SYSTEM.CATALOG for the table. The CREATE TABLE statement and SchemaTool will now use the above features when generating the DDL for column encoded tables with discontinuous qualifiers. This enables re-generating the Phoenix metadata for tables that have discontinous encoded column qualifers, so that the re-generated Phoenix tables will work on copied or replicated HBase data tables from the source database. For such tables, the generated CREATE TABLE statements will only be executable for Phoenix versions having this patch. However, systems that do not have this fix cannot correctly re-generate the table metadata. On older systems the only way to achieve HBase data table level compatibility is to manually edit SYSTEM.CATALOG.

    Description

      For column encoded tables,create table generate column qualifier order.

      When moving data between instances, and the original table had some columns removed, or swapped around, then the column qualifers in the table are not guarenteed to start from the expected value and increase by 1 for each column, based on the ordering of columns.

      This means that when we load a data table via HBase (i.e from a snaphsot), and then execute the DDL created by the show create table, or by other means, the column_qualifiers on the new table are going to point to the wrong Hbase cell.

      We need to accept and use COLUMN_QUALIFIER properties for columns,  and include them in in the show create table output for column encoded tables.

      We also need to accept and generate QUALIFIER_COUNTER for the tables.

      Attachments

        Issue Links

          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. PHOENIX-6764 Implement Binary and Hexadecimal literals

          • Major - Major loss of function.
          • Resolved
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. PHOENIX-6889 Improve extraction of ENCODED_QUALIFIERs

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #1553

          Web Link GitHub Pull Request #1556

          Activity

            People

              ameszaros Aron Attila Meszaros
              stoty Istvan Toth
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: