[HIVE-26375] Invalid materialized view after rebuild if source table was compacted

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-2
• Component/s: Materialized views, Transactions
• Labels: pull-request-available
• Assignee: Krisztian Kasa
• Reporter: Krisztian Kasa
• Created: 07/Jul/22 03:49
• Updated: 05/Apr/24 11:58
• Resolved: 22/Jul/22 10:19

Description

After HIVE-25656 MV state depends on the number of rows deleted/updated in the source tables of the view. However if one of the source tables are major compacted the delete delta files are no longer available and reproducing the rows should be deleted from the MV is no longer possible.

create table t1(a int, b varchar(128), c float) stored as orc TBLPROPERTIES ('transactional'='true');
insert into t1(a,b, c) values (1, 'one', 1.1), (2, 'two', 2.2), (NULL, NULL, NULL);
create materialized view mv1 stored as orc TBLPROPERTIES ('transactional'='true') as select a,b,c from t1 where a > 0 or a is null;
update t1 set b = 'Changed' where a = 1;
alter table t1 compact 'major';
alter materialized view t1 rebuild;
select * from mv1;

Select should result

      "1\tChanged\t1.1",
      "2\ttwo\t2.2",
      "NULL\tNULL\tNULL"

but was

      "1\tone\t1.1",      
      "2\ttwo\t2.2",
      "NULL\tNULL\tNULL",
      "1\tChanged\t1.1"

Issue Links

• relates to: HIVE-25656 Get materialized view state based on number of affected rows of transactions
• links to: GitHub Pull Request #3420