[HIVE-26296] RuntimeException when executing EXPLAIN CBO JOINCOST on query with JDBC tables

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-2
• Component/s: CBO, HiveServer2
• Labels: pull-request-available
• Target Version/s: 4.0.0-alpha-2
• Assignee: Stamatis Zampetakis
• Reporter: Stamatis Zampetakis
• Created: 07/Jun/22 14:50
• Updated: 16/Nov/22 13:50
• Resolved: 09/Jun/22 10:10

Description

Consider a JDBC database with two tables author and book.

CREATE EXTERNAL TABLE author
(
    id int,
    fname varchar(20),
    lname varchar(20)
)
STORED BY 'org.apache.hive.storage.jdbc.JdbcStorageHandler'
TBLPROPERTIES (
    "hive.sql.database.type" = "MYSQL",
    "hive.sql.jdbc.driver" = "com.mysql.jdbc.Driver",
    ...
    "hive.sql.table" = "author"
    );

CREATE EXTERNAL TABLE book
(
    id int,
    title varchar(100),
    author int
)
STORED BY 'org.apache.hive.storage.jdbc.JdbcStorageHandler'
TBLPROPERTIES (
    "hive.sql.database.type" = "MYSQL",
    "hive.sql.jdbc.driver" = "com.mysql.jdbc.Driver",
    ...
    "hive.sql.table" = "book"
    );

Executing an EXPLAIN CBO JOINCOST with a query joining two JDBC tables fails with RuntimeException while trying to compute the selectivity of the join.

EXPLAIN CBO JOINCOST 
SELECT a.lname, b.title FROM author a JOIN book b ON a.id=b.author;

Stacktrace

java.lang.RuntimeException: Unexpected Join type: org.apache.calcite.adapter.jdbc.JdbcRules$JdbcJoin
	at org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdSelectivity.computeInnerJoinSelectivity(HiveRelMdSelectivity.java:156)
	at org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdSelectivity.getSelectivity(HiveRelMdSelectivity.java:68)
	at GeneratedMetadataHandler_Selectivity.getSelectivity_$(Unknown Source)
	at GeneratedMetadataHandler_Selectivity.getSelectivity(Unknown Source)
	at org.apache.calcite.rel.metadata.RelMetadataQuery.getSelectivity(RelMetadataQuery.java:426)
	at org.apache.calcite.rel.metadata.RelMdUtil.getJoinRowCount(RelMdUtil.java:736)
	at org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:195)
	at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown Source)
	at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source)
	at org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:212)
	at org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:140)
	at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown Source)
	at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source)
	at org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:212)
	at org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:191)
	at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown Source)
	at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source)
	at org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:212)
	at org.apache.calcite.rel.externalize.RelWriterImpl.explain_(RelWriterImpl.java:100)
	at org.apache.calcite.rel.externalize.RelWriterImpl.done(RelWriterImpl.java:144)
	at org.apache.calcite.rel.AbstractRelNode.explain(AbstractRelNode.java:246)
	at org.apache.calcite.plan.RelOptUtil.toString(RelOptUtil.java:2308)
	at org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:648)
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12699)
	at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:460)
	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:317)
	at org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:180)
	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:317)
	at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:224)
	at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:106)
	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:495)
	at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:447)
	at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:412)
	at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:406)
	at org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:121)
	at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:227)
	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
	at org.apache.hadoop.hive.cli.CliDriver.processCmd1(CliDriver.java:200)
	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:126)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:421)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:352)

Issue Links

• links to: GitHub Pull Request #3349