[HIVE-14807] analyze table compute statistics fails due to presence of Infinity value in double column

• Type: Sub-task
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.2.1
• Fix Version/s: 2.3.0

Environment

HDP 2.4 on CentOS 7 Linux

• Assignee: Pengcheng Xiong
• Reporter: Edi Bice
• Created: 21/Sep/16 20:47
• Updated: 20/Apr/17 17:35
• Resolved: 20/Apr/17 17:35

Description

2016-09-21 20:27:45,074 ERROR [pool-3-thread-12]: metastore.RetryingHMSHandler (RetryingHMSHandler.java:invoke(173)) - Retrying HMSHandler after 2000 ms (attempt 5 of 10) with error: javax.jdo.JDODataStoreException: Cannot set Object parameter: value = Infinity for column "`TAB_COL_STATS`.`DOUBLE_HIGH_VALUE`" : 'Infinity' is not a valid numeric or approximate numeric value
at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:732)
at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752)
at org.apache.hadoop.hive.metastore.ObjectStore.writeMTableColumnStatistics(ObjectStore.java:5937)
at org.apache.hadoop.hive.metastore.ObjectStore.updateTableColumnStatistics(ObjectStore.java:5994)
at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
at com.sun.proxy.$Proxy2.updateTableColumnStatistics(Unknown Source)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.update_table_column_statistics(HiveMetaStore.java:4358)
at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
at com.sun.proxy.$Proxy4.update_table_column_statistics(Unknown Source)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$update_table_column_statistics.getResult(ThriftHiveMetastore.java:10701)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$update_table_column_statistics.getResult(ThriftHiveMetastore.java:10685)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:110)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:106)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:118)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
NestedThrowablesStackTrace:
java.sql.SQLException: 'Infinity' is not a valid numeric or approximate numeric value
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1078)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:989)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:975)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:920)
at com.mysql.jdbc.PreparedStatement.setDouble(PreparedStatement.java:3743)
at com.jolbox.bonecp.PreparedStatementHandle.setDouble(PreparedStatementHandle.java:788)
at org.datanucleus.store.rdbms.ParamLoggingPreparedStatement.setDouble(ParamLoggingPreparedStatement.java:726)
at org.datanucleus.store.rdbms.mapping.datastore.DoubleRDBMSMapping.setObject(DoubleRDBMSMapping.java:264)
at org.datanucleus.store.rdbms.mapping.java.SingleFieldMapping.setObject(SingleFieldMapping.java:211)
at org.datanucleus.store.rdbms.fieldmanager.ParameterSetter.storeObjectField(ParameterSetter.java:197)
at org.datanucleus.state.JDOStateManager.providedObjectField(JDOStateManager.java:1269)
at org.apache.hadoop.hive.metastore.model.MTableColumnStatistics.jdoProvideField(MTableColumnStatistics.java)
at org.apache.hadoop.hive.metastore.model.MTableColumnStatistics.jdoProvideFields(MTableColumnStatistics.java)
at org.datanucleus.state.JDOStateManager.provideFields(JDOStateManager.java:1346)
at org.datanucleus.store.rdbms.request.InsertRequest.execute(InsertRequest.java:289)
at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.insertTable(RDBMSPersistenceHandler.java:167)