[IMPALA-12547] Set the default worker threads for compaction to 1

• Type: Wish
• Status: In Progress
• Priority: Major
• Resolution: Unresolved
• Component/s: fe
• Epic Color: ghx-label-13
• Assignee: Sai Hemanth Gantasala
• Reporter: Sai Hemanth Gantasala
• Created: 08/Nov/23 00:31
• Updated: 07/Dec/23 19:38

Description

Set the default worker threads for compaction to 1.

Currently worker threads for compaction to 4, as a result, in the external frontend mode, there are a lot of calls to HMS in idle state. So, it is flooding the catalogD log with unnecessary calls to HMS.

I0503 14:13:26.944278 15874 CatalogMetastoreServer.java:215] Invoking HMS API: get_current_notificationEventId
I0503 14:13:26.948185 15874 CatalogMetastoreServer.java:215] Invoking HMS API: get_config_value
I0503 14:13:26.949983 15874 CatalogMetastoreServer.java:215] Invoking HMS API: get_next_notification
I0503 14:13:41.494956 15300 CatalogMetastoreServer.java:215] Invoking HMS API: find_next_compact2
I0503 14:13:41.501145 15303 CatalogMetastoreServer.java:215] Invoking HMS API: find_next_compact2
I0503 14:13:41.507638 15306 CatalogMetastoreServer.java:215] Invoking HMS API: find_next_compact2
I0503 14:13:41.541407 15309 CatalogMetastoreServer.java:215] Invoking HMS API: find_next_compact2

After the above-proposed values for the configs, this is the log info in an idle state.

I1107 15:58:03.130424 533568 CatalogMetastoreServer.java:215] Invoking HMS API: get_next_notification
I1107 15:58:03.157773 533460 CatalogMetastoreServer.java:215] Invoking HMS API: scheduled_query_poll
I1107 15:58:03.178020 533480 CatalogMetastoreServer.java:215] Invoking HMS API: find_next_compact2