Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-7365

Failed to read column added to existing Hive partition

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Storage - Hive
    • Labels:
      None

      Description

      Prerequisities:

      Enable ACID in Hive https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions.

      Steps to reproduce:

      1) create table hive_bucketed2 (emp_id int, first_name string) PARTITIONED BY (`col_year_month` string) clustered by (emp_id) into 4 buckets stored as orc tblproperties ('transactional'='true');
      2) insert into hive_bucketed2 PARTITION (col_year_month = '2019-09') values (1, 'A'),(2, 'B');
      3) alter table hive_bucketed2 add columns (age INT);
      4) insert into hive_bucketed2 PARTITION (col_year_month = '2019-09') values (11, '1A', 10),(12, '1B', 22);
      5) select * from hive.hive_bucketed2;

      Workaround (may be a little bit risky :

      1. Connect to Hive metastore database.

      https://analyticsanvil.files.wordpress.com/2016/08/hive_metastore_database_diagram.png

      2. Find SDS linked to desired PARTITIONS . Actually you need CD_ID's for such SDS.

      3. Insert your column into COLUMNS_V2 with CD_ID found at previous step.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              IhorHuzenko Igor Guzenko
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: