[CALCITE-624] IllegalArgumentException: duplicate key in ReflectiveRelMetadataProvider

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Cannot Reproduce
• Assignee: Unassigned
• Reporter: Jesús Camacho Rodríguez
• Created: 16/Mar/15 19:30
• Updated: 27/Feb/24 22:23
• Resolved: 16/Mar/15 19:31

Description

We have our own implementation of RelMDParallelism provider in Hive: HiveRelMDParallelism. It has two methods for the HiveJoin class: isPhaseTransition and splitCount. We hit the problem shown in the stacktrace, that we have not been able to reproduce again.

java.lang.IllegalArgumentException: duplicate key: class org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveJoin
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
        at com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:67)
        at com.google.common.collect.ImmutableMap.copyOf(ImmutableMap.java:303)
        at org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider.reflectiveSource(ReflectiveRelMetadataProvider.java:204)
        at org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider.reflectiveSource(ReflectiveRelMetadataProvider.java:112)
        at org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdParallelism.getMetadataProvider(HiveRelMdParallelism.java:44)
        at org.apache.hadoop.hive.ql.optimizer.calcite.HiveDefaultRelMetadataProvider.getMetadataProvider(HiveDefaultRelMetadataProvider.java:52)
        at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:760)
        at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:713)
        at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:109)
        at org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:730)
        at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:145)
        at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:105)
        at org.apache.hadoop.hive.ql.parse.CalcitePlanner.getOptimizedAST(CalcitePlanner.java:580)
...

We realized that the same class was ending up twice in a Set when the methods are added for reflection calls.

Issue Links

• is related to: CALCITE-632 Sort order returned by SUPERCLASS_COMPARATOR in ReflectiveRelMetadataProvider is inconsistent