[IMPALA-12468] Add the ability to update EventProcessorStatus

• Type: Improvement
• Status: In Progress
• Priority: Minor
• Resolution: Unresolved
• Component/s: be, Catalog, fe
• Epic Link: event-processor-supportability
• Epic Color: ghx-label-11
• Assignee: Maxwell Guo
• Reporter: Maxwell Guo
• Created: 27/Sep/23 05:40
• Updated: 12/Mar/24 09:15

Description

Once the impala and hive's status is missmatched , and the EventProcessorStatus become NEED_INVALIDATE， we usually use invalidate metadata to reset the catalog instance. And then impala will update the status to ACTIVE .

But if impala contains many tables , the cost of invalidate is a bit high for a global invalidate. So we may invalidate metadata for tables one by one for these incremental changed table. For example , we have 1000,000,000,000 tables but only some of the table event process occurs CatalogException and MetastoreNotificationNeedsInvalidateException was thrown. I think there is no need to invalidate all table caches in order to reset the catalog instance see here .

MetaStoresProcessor 's async update process will not update the currentStatus when the status is not ACTIVE, see here

So what about add a new SQL grammar : SET EVENT STATUS ${status} ?

Issue Links

• is a parent of: IMPALA-12506 Add the ability to update EventProcessorStatus through webUi