Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-24994

get_aggr_stats_for call fail with "Tried to send an out-of-range integer"

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 4.0.0
    • Component/s: Hive

      Description

      aggrColStatsForPartitions call fail with the Postgres LIMIT if the no of partitions passed in the direct sql goes beyond the 32767

      postgresql.util.PSQLException: An I/O error occurred while sending to the backend.
                       at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:337) ~[postgresql-42.2.8.jar:42.2.8]
                       at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:446) ~[postgresql-42.2.8.jar:42.2.8]
                       at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:370) ~[postgresql-42.2.8.jar:42.2.8]
                       at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:149) ~[postgresql-42.2.8.jar:42.2.8]
                       at org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:108) ~[postgresql-42.2.8.jar:42.2.8]
                       at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeQuery(ProxyPreparedStatement.java:52) ~[HikariCP-2.6.1.jar:?]
                       at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeQuery(HikariProxyPreparedStatement.java) [HikariCP-2.6.1.jar:?]
                       at org.datanucleus.store.rdbms.ParamLoggingPreparedStatement.executeQuery(ParamLoggingPreparedStatement.java:375) [datanucleus-rdbms-4.1.19.jar:?]
                       at org.datanucleus.store.rdbms.SQLController.executeStatementQuery(SQLController.java:552) [datanucleus-rdbms-4.1.19.jar:?]
                       at org.datanucleus.store.rdbms.query.SQLQuery.performExecute(SQLQuery.java:645) [datanucleus-rdbms-4.1.19.jar:?]
                       at org.datanucleus.store.query.Query.executeQuery(Query.java:1855) [datanucleus-core-4.1.17.jar:?]
                       at org.datanucleus.store.rdbms.query.SQLQuery.executeWithArray(SQLQuery.java:807) [datanucleus-rdbms-4.1.19.jar:?]
                       at org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:368) [datanucleus-api-jdo-4.2.4.jar:?]
                       at org.datanucleus.api.jdo.JDOQuery.executeWithArray(JDOQuery.java:267) [datanucleus-api-jdo-4.2.4.jar:?]
                       at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.executeWithArray(MetaStoreDirectSql.java:2058) [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
                       at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.executeWithArray(MetaStoreDirectSql.java:2050) [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
                       at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.access$1500(MetaStoreDirectSql.java:110) [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
                       at org.apache.hadoop.hive.metastore.MetaStoreDirectSql$15$1.run(MetaStoreDirectSql.java:1530) [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
                       at org.apache.hadoop.hive.metastore.Batchable.runBatched(Batchable.java:73) [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
                       at org.apache.hadoop.hive.metastore.MetaStoreDirectSql$15.run(MetaStoreDirectSql.java:1521) [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
                       at org.apache.hadoop.hive.metastore.Batchable.runBatched(Batchable.java:73) [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
                       at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.partsFoundForPartitions(MetaStoreDirectSql.java:1518) [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
                       at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.aggrColStatsForPartitions(MetaStoreDirectSql.java:1489) [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
                       at org.apache.hadoop.hive.metastore.ObjectStore$20.getSqlResult(ObjectStore.java:8966) [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
                       at org.apache.hadoop.hive.metastore.ObjectStore$20.getSqlResult(ObjectStore.java:8962) [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
                       at org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:3757) [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
                       at org.apache.hadoop.hive.metastore.ObjectStore.get_aggr_stats_for(ObjectStore.java:8981) [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
                       at org.apache.hadoop.hive.metastore.ObjectStore.get_aggr_stats_for(ObjectStore.java:8951) [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
                       at sun.reflect.GeneratedMethodAccessor65.invoke(Unknown Source) ~[?:?]
                       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_261]
                       at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_261]
                       at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97) [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
                       at com.sun.proxy.$Proxy37.get_aggr_stats_for(Unknown Source) [?:?]
                       at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_aggr_stats_for(HiveMetaStore.java:8254) [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
                       at sun.reflect.GeneratedMethodAccessor64
               Caused by: java.io.IOException: Tried to send an out-of-range integer as a 2-byte value: 83666
              at org.postgresql.core.PGStream.sendInteger2(PGStream.java:266) ~[postgresql-42.2.8.jar:42.2.8]
              at org.postgresql.core.v3.QueryExecutorImpl.sendParse(QueryExecutorImpl.java:1492) ~[postgresql-42.2.8.jar:42.2.8]
              at org.postgresql.core.v3.QueryExecutorImpl.sendOneQuery(QueryExecutorImpl.java:1815) ~[postgresql-42.2.8.jar:42.2.8]
              at org.postgresql.core.v3.QueryExecutorImpl.sendQuery(QueryExecutorImpl.java:1378) ~[postgresql-42.2.8.jar:42.2.8]
              at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:302) ~[postgresql-42.2.8.jar:42.2.8]
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Rajkumar Singh Rajkumar Singh
                Reporter:
                Rajkumar Singh Rajkumar Singh
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 0.5h
                  0.5h