[IMPALA-11181] Improving performance of compaction checking

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.1.0
• Component/s: Catalog
• Epic Color: ghx-label-9
• Assignee: Yu-Wen Lai
• Reporter: Yu-Wen Lai
• Created: 15/Mar/22 00:01
• Updated: 31/Mar/22 22:09
• Resolved: 31/Mar/22 22:09

Description

After HIVE-25753, we don't need to explicitly set all partitions names for a table. Furthermore, we can send the last compaction id over to HMS and HMS will only send back information if there any newer compaction. In this way, we should be able to avoid unnecessary data be sent between HMS and Catalogd.