[IMPALA-10737] Optimize Iceberg metadata handling

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.1.0
• Component/s: Frontend
• Labels: impala-iceberg
• Epic Link: Iceberg support in Impala
• Epic Color: ghx-label-12
• Assignee: Tamas Mate
• Reporter: Zoltán Borók-Nagy
• Created: 08/Jun/21 17:24
• Updated: 05/Apr/22 07:43
• Resolved: 04/Mar/22 15:31

Description

Currently we re-read Iceberg table metadata in several cases.

We should rather keep it in memory and use it when possible.

Also, when refreshing a table we should use Iceberg's refresh() API to avoid unnecessary re-reads of manifest files:

https://github.com/apache/iceberg/blob/282b6f9f1cae8d4fd5ff7c73de513ca91f01fddc/core/src/main/java/org/apache/iceberg/TableOperations.java#L45

 

Issue Links

• incorporates: IMPALA-11170 Remove Iceberg BaseTable serialization