[HIVE-12976] MetaStoreDirectSql doesn't batch IN lists in all cases - ASF JIRA

Log work

Agile Board

Rank to Top

Rank to Bottom

Bulk Copy Attachments

Bulk Move Attachments

Voters

Watch issue

Watchers

Create sub-task

Convert to sub-task

Move

Link

Clone

Labels

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 2.1.0
Component/s: None
Labels:
None

Description

That means that some RDBMS products with arbitrary limits cannot run these queries. I hope HBase metastore comes soon and delivers us from Oracle! For now, though, we have to fix this.

hive> select * from lineitem where l_orderkey = 1212000001;

Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The incoming request has too many parameters. The server supports a maximum of 2100 parameters. Reduce the number of parameters and resend the request.
        at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:215) ~[sqljdbc41.jar:?]
        at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1635) ~[sqljdbc41.jar:?]
        at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(SQLServerPreparedStatement.java:426) ~[sqljdbc41.jar:?]

        at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543) ~[datanucleus-api-jdo-4.2.1.jar:?]
        at org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:388) ~[datanucleus-api-jdo-4.2.1.jar:?]
        at org.datanucleus.api.jdo.JDOQuery.executeWithArray(JDOQuery.java:264) ~[datanucleus-api-jdo-4.2.1.jar:?]
        at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.executeWithArray(MetaStoreDirectSql.java:1681) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.partsFoundForPartitions(MetaStoreDirectSql.java:1266) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.aggrColStatsForPartitions(MetaStoreDirectSql.java:1196) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.metastore.ObjectStore$9.getSqlResult(ObjectStore.java:6742) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.metastore.ObjectStore$9.getSqlResult(ObjectStore.java:6738) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:2525) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.metastore.ObjectStore.get_aggr_stats_for(ObjectStore.java:6738) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]