[HIVE-18776] MaterializationsInvalidationCache loading causes race condition in the metastore

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Component/s: Materialized views, Metastore
• Assignee: Jesús Camacho Rodríguez
• Reporter: Alan Gates
• Created: 22/Feb/18 22:16
• Updated: 27/Feb/24 22:23
• Resolved: 27/Feb/18 23:52

Description

I am seeing occasional failures running metastore tests where operations are failing saying that there is no open transaction.  I have traced this to a race condition in loading the materialized view invalidation cache.  When it is initialized (either in HiveMetaStoreClient in embedded mode or in HiveMetaStore in remote mode) it grabs a copy of the current RawStore instance and then loads the cache in a separate thread.  But ObjectStore keeps state regarding JDO transactions with the underlying RDBMS.  So with the loader thread and the initial thread both doing operations against the RawStore they sometimes mess up each others transaction stack.  In a quick test I used HMSHandler.newRawStoreForConf() to fix this, which seemed to work.

A reference to the TxnHandler is also called.  I suspect this will run into a similar issue.

Issue Links

• is related to: HIVE-17751 Separate HMS Client and HMS server into separate sub-modules