Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
1.21.0, 1.24.0
-
None
-
None
Description
The RelDistribution is calculated by:
1. Get the Project input's RelDistribution. If it is a TableScan which scans a partitioned table the returned RelDistribution object contains the indexes of partition columns like: RelDistribution(hash[0, 4])
2. If the project contains these columns we need a mapping to map the key indexes in the RelDistribution.
Mapping is generated here:
https://github.com/apache/calcite/blob/b3b3e06eef7eeedf2de32b44d061af7867633be6/core/src/main/java/org/apache/calcite/rel/metadata/RelMdDistribution.java#L164
This mapping is an INVERSE_FUNCTION
https://github.com/apache/calcite/blob/2088488ac8327b19512a76a122cae2961fc551c3/core/src/main/java/org/apache/calcite/rel/core/Project.java#L375
which does not support getTargetOpt
Stack trace from UT ( RelMdDistributionTest.java ) I created to reproduce the issue in Calcite:
[0;31;1mFAILURE[0m [0;1m 2.1sec[0m, org.apache.calcite.rel.metadata.[0;1mRelMdDistributionTest[0m > [0;1mtest()[0m java.lang.UnsupportedOperationException at org.apache.calcite.util.mapping.Mappings$AbstractMapping.getSourceOpt(Mappings.java:913) at org.apache.calcite.util.mapping.Mappings$InverseMapping.getTargetOpt(Mappings.java:1800) at org.apache.calcite.rel.RelDistributions$RelDistributionImpl.apply(RelDistributions.java:145) at org.apache.calcite.rel.metadata.RelMdDistribution.project(RelMdDistribution.java:166) at org.apache.calcite.rel.metadata.RelMdDistribution.distribution(RelMdDistribution.java:98) [0;1mat org.apache.calcite.rel.metadata.RelMdDistributionTest.test(RelMdDistributionTest.java:76)
And a stacktrace from Hive where we hit this issue: cbo_query74_stacktrace.txt