[IMPALA-12809] Iceberg metadata table scanner should always be scheduled to the coordinator

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 4.4.0
• Component/s: Backend
• Labels: impala-iceberg
• Epic Color: ghx-label-14
• Assignee: Daniel Becker
• Reporter: Tamas Mate
• Created: 12/Feb/24 09:46
• Updated: 16/May/24 14:12
• Resolved: 29/Mar/24 13:47

Description

On larger clusters the Iceberg metadata scanner can be scheduled to executors, for example during a join. The fragment in this case will fail a precondition check, because either the frontend_ object will not be present or the table. Setting exec_at_coord to true is not enough and these fragments should be scheduled to the coord_only_executor_group.

Additionally, setting NUM_NODES=1 should be a viable workaround.

Reproducible with the following local dev Impala cluster:
./bin/start-impala-cluster.py --cluster_size=3 --num_coordinators=1 --use_exclusive_coordinators
and query:
select count(b.parent_id) from functional_parquet.iceberg_query_metadata.history a
join functional_parquet.iceberg_query_metadata.history b on a.snapshot_id = b.snapshot_id;

Issue Links

• relates to: IMPALA-10947 SQL support for querying Iceberg metadata