[FLINK-26362] "IndexOutOfBoundsException" when subquery select all field from using hive dialect

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.16.0
• Component/s: Connectors / Hive
• Labels: pull-request-available
• Assignee: luoyuxia
• Reporter: luoyuxia
• Created: 25/Feb/22 02:21
• Updated: 05/Aug/22 12:02
• Resolved: 05/Aug/22 12:01

Description

With hive dialect, can be reproduced using following code:

 tableEnv.executeSql("CREATE TABLE t1 (c1 INT, c2 CHAR(100))");
 tableEnv.executeSql("CREATE TABLE t2 (c1 INT)");
List<Row> results = CollectionUtil.iteratorToList(tableEnv.executeSql("SELECT c1 FROM t1 WHERE c1 IN (SELECT c1 FROM t2)").
                        collect());

Then it will throw IndexOutOfBoundsException: 0, the reason is from the following comments:
If it's a subquery and the project is identity, we skip creating this project. This is to handle an issue with calcite SubQueryRemoveRule. The rule checks col uniqueness by calling RelMetadataQuery::areColumnsUnique with an empty col set, which always returns null for a project and thus introduces unnecessary agg node.

So there could be no project node and only tablescan node in subquery. Then when we try to do type conversion for the subquery, with the following code, it'll throw exception when there's no project node.

if (queryRelNode instanceof Project) {
  return replaceProjectForTypeConversion(
                    rexBuilder,
                    (Project) queryRelNode,
                    targetCalcTypes,
                    targetHiveTypes,
                    funcConverter);
} else {
  RelNode newInput =
                        addTypeConversions(
                                rexBuilder,
                                queryRelNode.getInput(0),
                                targetCalcTypes,
                                targetHiveTypes,
                                funcConverter);
                queryRelNode.replaceInput(0, newInput);
                return queryRelNode;
}

Issue Links

• links to: GitHub Pull Request #18937