Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-12976

MetaStoreDirectSql doesn't batch IN lists in all cases

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.1.0
    • Component/s: None
    • Labels:
      None

      Description

      That means that some RDBMS products with arbitrary limits cannot run these queries. I hope HBase metastore comes soon and delivers us from Oracle! For now, though, we have to fix this.

      hive> select * from lineitem where l_orderkey = 1212000001;
      
      Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The incoming request has too many parameters. The server supports a maximum of 2100 parameters. Reduce the number of parameters and resend the request.
              at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:215) ~[sqljdbc41.jar:?]
              at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1635) ~[sqljdbc41.jar:?]
              at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(SQLServerPreparedStatement.java:426) ~[sqljdbc41.jar:?]
      
              at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543) ~[datanucleus-api-jdo-4.2.1.jar:?]
              at org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:388) ~[datanucleus-api-jdo-4.2.1.jar:?]
              at org.datanucleus.api.jdo.JDOQuery.executeWithArray(JDOQuery.java:264) ~[datanucleus-api-jdo-4.2.1.jar:?]
              at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.executeWithArray(MetaStoreDirectSql.java:1681) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
              at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.partsFoundForPartitions(MetaStoreDirectSql.java:1266) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
              at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.aggrColStatsForPartitions(MetaStoreDirectSql.java:1196) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
              at org.apache.hadoop.hive.metastore.ObjectStore$9.getSqlResult(ObjectStore.java:6742) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
              at org.apache.hadoop.hive.metastore.ObjectStore$9.getSqlResult(ObjectStore.java:6738) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
              at org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:2525) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
              at org.apache.hadoop.hive.metastore.ObjectStore.get_aggr_stats_for(ObjectStore.java:6738) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
      

        Attachments

        1. HIVE-12976.patch
          21 kB
          Sergey Shelukhin
        2. HIVE-12976.01.patch
          23 kB
          Sergey Shelukhin
        3. HIVE-12976.02.patch
          23 kB
          Sergey Shelukhin

          Issue Links

            Activity

              People

              • Assignee:
                sershe Sergey Shelukhin
                Reporter:
                gopalv Gopal Vijayaraghavan
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: