[IMPALA-12509] Optimize the backend startup and planner time of large Iceberg table query - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: Impala 4.4.0
Component/s: None
Labels:
- impala-iceberg

Epic Color:
ghx-label-11

Description

We found that when querying an Iceberg table with a large number of files (>=200000), the Query Plan and start backends took an abnormal time (>= 2s). The reason was that unnecessary objects were serialized when building TQueryCtx. The main function involved is IcebergTable::toThriftDescriptor

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

image-2023-10-26-15-34-28-254.png
26/Oct/23 07:34
103 kB
Fu Lili

Activity

People

Assignee:: Fu Lili

Reporter:: Fu Lili

Votes:: 0 Vote for this issue

Watchers:: 4 Start watching this issue

Dates

Created:: 20/Oct/23 11:00

Updated:: 02/Nov/23 03:58

Resolved:: 02/Nov/23 03:58