[HIVE-12976] MetaStoreDirectSql doesn't batch IN lists in all cases

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.1.0
• Assignee: Sergey Shelukhin
• Reporter: Gopal Vijayaraghavan
• Created: 01/Feb/16 19:03
• Updated: 21/Jun/16 15:43
• Resolved: 06/Feb/16 00:14

Description

That means that some RDBMS products with arbitrary limits cannot run these queries. I hope HBase metastore comes soon and delivers us from Oracle! For now, though, we have to fix this.

hive> select * from lineitem where l_orderkey = 1212000001;

Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The incoming request has too many parameters. The server supports a maximum of 2100 parameters. Reduce the number of parameters and resend the request.
        at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:215) ~[sqljdbc41.jar:?]
        at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1635) ~[sqljdbc41.jar:?]
        at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(SQLServerPreparedStatement.java:426) ~[sqljdbc41.jar:?]

        at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543) ~[datanucleus-api-jdo-4.2.1.jar:?]
        at org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:388) ~[datanucleus-api-jdo-4.2.1.jar:?]
        at org.datanucleus.api.jdo.JDOQuery.executeWithArray(JDOQuery.java:264) ~[datanucleus-api-jdo-4.2.1.jar:?]
        at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.executeWithArray(MetaStoreDirectSql.java:1681) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.partsFoundForPartitions(MetaStoreDirectSql.java:1266) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.aggrColStatsForPartitions(MetaStoreDirectSql.java:1196) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.metastore.ObjectStore$9.getSqlResult(ObjectStore.java:6742) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.metastore.ObjectStore$9.getSqlResult(ObjectStore.java:6738) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:2525) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.metastore.ObjectStore.get_aggr_stats_for(ObjectStore.java:6738) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]

Issue Links

• links to: RB