[HIVE-24403] change min_history_level schema change to be compatible with previous version

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Component/s: Metastore
• Labels: pull-request-available
• Assignee: Peter Varga
• Reporter: Peter Varga
• Created: 17/Nov/20 16:35
• Updated: 17/Nov/22 08:47
• Resolved: 04/Dec/20 09:13

Description

In some configurations the HMS backend DB is used by HMS services with different versions. 
HIVE-23107 dropped the min_history_level table from the backend DB making the new schema version incompatible with the older HMS services. 
It is possible to modify that change to keep the compatibility

• Keep the min_history_level table
• Add the new fields for the compaction_queue the same way
• Create a feature flag for min_history_level and if it is on
• Keep the logic inserting to the table during openTxn
• Keep the logic removing the records at commitTxn and abortTxn
• Change the logic in the cleaner, to get the highwatermark the old way
• But still change it to not start the cleaning before that

• The txn_to_write_id table cleaning can work the new way in the new version and the old way in the old version
• This feature flag can be automatically setup based on the existence of the min_history level table, this way if the table will be dropped all HMS-s can switch to the new functionality without restart

Issue Links

• fixes: HIVE-23107 Remove MIN_HISTORY_LEVEL table
• links to: GitHub Pull Request #1688