Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-19891

inserting into external tables with custom partition directories may cause data loss

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.0.0
    • Component/s: None
    • Labels:
      None

      Description

      tbl1 is just used as a prop to create data, could be an existing directory for an external table.
      Due to weird behavior of LoadTableDesc (some ancient code for overriding old partition path), custom partition path is overwritten after the query and the data in it ceases being a part of the table (can be seen in desc formatted output with masking commented out in QTestUtil)
      This affects branch-1 too, so it's pretty old.

      drop table tbl1;
      CREATE TABLE tbl1 (index int, value int ) PARTITIONED BY ( created_date string );
      insert into tbl1 partition(created_date='2018-02-01') VALUES (2, 2);
      
      CREATE external TABLE tbl2 (index int, value int ) PARTITIONED BY ( created_date string );
      ALTER TABLE tbl2 ADD PARTITION(created_date='2018-02-01');
      ALTER TABLE tbl2 PARTITION(created_date='2018-02-01') SET LOCATION 'file:/Users/sergey/git/hivegit/itests/qtest/target/warehouse/tbl1/created_date=2018-02-01';
      select * from tbl2;
      describe formatted tbl2 partition(created_date='2018-02-01');
      insert into tbl2 partition(created_date='2018-02-01') VALUES (1, 1);
      select * from tbl2;
      describe formatted tbl2 partition(created_date='2018-02-01');
      

        Attachments

        1. HIVE-19891.patch
          14 kB
          Sergey Shelukhin
        2. HIVE-19891.01.patch
          14 kB
          Sergey Shelukhin
        3. HIVE-19891.02.patch
          14 kB
          Sergey Shelukhin
        4. HIVE-19891.03.patch
          14 kB
          Sergey Shelukhin
        5. HIVE-19891.04.patch
          14 kB
          Sergey Shelukhin
        6. HIVE-19891.05.patch
          14 kB
          Sergey Shelukhin
        7. HIVE-19891.06.patch
          14 kB
          Sergey Shelukhin
        8. HIVE-19891.07.patch
          14 kB
          Sergey Shelukhin

          Issue Links

            Activity

              People

              • Assignee:
                sershe Sergey Shelukhin
                Reporter:
                sershe Sergey Shelukhin
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: