[IMPALA-12197] Time Travel Queries on Iceberg tables fail with "ERROR: IllegalArgumentException: null"

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 2.13.0
• Fix Version/s: Impala 4.3.0
• Epic Color: ghx-label-12
• Assignee: Andrew Sherman
• Reporter: Andrew Sherman
• Created: 09/Jun/23 00:02
• Updated: 17/Jun/23 00:22
• Resolved: 17/Jun/23 00:22

Description

When using local catalog mode, if a runtime filter is being generated for a time travel iceberg table, then a query may fail with "ERROR: IllegalArgumentException: null"

In the planner an Iceberg table that is being accessed with Time Travel is represented by an IcebergTimeTravelTable object. This object represents a time-based variation on a base table. The  IcebergTimeTravelTable may represent a different schema from the base table, it does this by tracking its own set of Columns. As part of generating a runtime filter the isClusteringColumn() method is called on the table. IcebergTimeTravelTable was delegating this call to the base object. In local catalog mode this method is implemented by LocalTable which has a Preconditions check (an assertion) that the column parameter matches the stored column. In this case the check fails as the base table and time travel table have their own distinct set of column objects.

A possible fix is to have IcebergTimeTravelTable provide its own isClusteringColumn() method. For iceberg there are no clustering columns so this method simply returns false.

I0608 18:19:04.009989 26712 jni-util.cc:288] 30478706c6b2ab5b:ac06893500000000] java.lang.IllegalArgumentException
    at com.google.common.base.Preconditions.checkArgument(Preconditions.java:131)
    at org.apache.impala.catalog.local.LocalTable$ColumnMap.isClusteringColumn(LocalTable.java:427)
    at org.apache.impala.catalog.local.LocalTable.isClusteringColumn(LocalTable.java:286)
    at org.apache.impala.catalog.local.LocalIcebergTable.isClusteringColumn(LocalIcebergTable.java:59)
    at org.apache.impala.catalog.ForwardingFeIcebergTable.isClusteringColumn(IcebergTimeTravelTable.java:546)
    at org.apache.impala.catalog.IcebergTimeTravelTable.isClusteringColumn(IcebergTimeTravelTable.java:70)
    at org.apache.impala.planner.RuntimeFilterGenerator.isBoundByPartitionColumns(RuntimeFilterGenerator.java:1088)
    at org.apache.impala.planner.RuntimeFilterGenerator.assignRuntimeFilters(RuntimeFilterGenerator.java:964)
    at org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:863)
    at org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:866)
    at org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:866)
    at org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:866)
    at org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:856)
    at org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:866)
    at org.apache.impala.planner.RuntimeFilterGenerator.generateRuntimeFilters(RuntimeFilterGenerator.java:735)
    at org.apache.impala.planner.Planner.createPlanFragments(Planner.java:147)
    at org.apache.impala.planner.Planner.createPlans(Planner.java:251)
    at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1772)
    at org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:2603)
    at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:2432)
    at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:2056)
    at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1831)
    at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:164)