[HIVE-22116] MaterializedView refresh check might return incorrect result when Compaction is run

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Component/s: Materialized views
• Assignee: Jesús Camacho Rodríguez
• Reporter: Peter Vary
• Created: 15/Aug/19 15:21
• Updated: 27/Feb/24 22:23
• Resolved: 11/Oct/19 17:16

Description

Reading the code of TxnHandler.getMaterializationInvalidationInfo I see that we decide on the freshness of the view based on the COMPLETED_TXN_COMPONENTS table. 
See: https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L2021
On the other hand if we run a major compaction we clean up COMPLETED_TXN_COMPONENTS table, so we lose all previous information. We do it in CompactionTxnHandler.markCleaned.
See: https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/CompactionTxnHandler.java#L382
 
When the following sequence of events happen we do not refresh the materialized view:

• Create Table
• Create MV
• Refresh MV
• Update Table
• Start major compaction
• Wait until compacted, and cleaned
• Select Table

Issue Links

• Is contained by: HIVE-21344 CBO: Reduce compilation time in presence of materialized views