Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-21598

CTAS on ACID table during incremental does not replicate data

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: HiveServer2, repl
    • Labels:

      Description

      Scenario

      create database dumpdb with dbproperties('repl.source.for'='1,2,3');

      use dumpdb;

      create table t1 (id int) clustered by(id) into 3 buckets stored as orc tblproperties ("transactional"="true");

      insert into t1 values(1);

      insert into t1 values(2);

      repl dump dumpdb;

      repl load loaddb from <bootstrap load directory>;

      use loaddb;

      select * from t1;

      --------

      t6.id

      --------

      1
      2

      +--------

      use dumpdb;

      create table t6 stored as orc tblproperties ("transactional"="true") as select * from t1;

      select * from t6;

      --------

      t6.id

      --------

      1
      2

      --------

      repl dump dumpdb from <last repl id>

      repl load loaddb from <inc dump location>;

      use loaddb;

      select * from t6;

      --------

      t6.id

      --------
      --------

      t6 gets created but there's no data.

       

      On further investigation, I see that the CommitTxnEvent's dump directory has _files but it is empty. Looks like we do not log names of the files created as part of CTAS.

        Attachments

        1. HIVE-21598.01.patch
          72 kB
          Ashutosh Bapat
        2. HIVE-21598.02.patch
          77 kB
          Ashutosh Bapat
        3. HIVE-21598.03.patch
          77 kB
          Ashutosh Bapat
        4. HIVE-21598.04.patch
          77 kB
          Ashutosh Bapat
        5. HIVE-21598.05.patch
          77 kB
          Ashutosh Bapat
        6. HIVE-21598.06.patch
          77 kB
          Ashutosh Bapat

          Issue Links

            Activity

              People

              • Assignee:
                ashutosh.bapat Ashutosh Bapat
                Reporter:
                ashutosh.bapat Ashutosh Bapat
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: