Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-24982

HMS- Postgres: Create table fail if SERDEPROPERTIES contains the NULL character

Log workAgile BoardRank to TopRank to BottomAdd voteVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 4.0.0
    • Component/s: Hive
    • Labels:
      None

      Description

      Postgres does not expect the NULL char ('\u0000') during the insert (ref: https://www.postgresql.org/message-id/1171970019.3101.328.camel%40coppola.muc.ecircle.de ) so create table with following SERDEPROPERTIES will fail with exception.

      ROW FORMAT SERDE 
        'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
      WITH SERDEPROPERTIES ( 
        'field.delim'='\u0000', 
        'serialization.format'='\u0000') 
      
      org.datanucleus.store.rdbms.exceptions.MappedDatastoreException: INSERT INTO "SERDE_PARAMS" ("PARAM_VALUE","SERDE_ID","PARAM_KEY") VALUES (?,?,?)
              at org.datanucleus.store.rdbms.scostore.JoinMapStore.internalPut(JoinMapStore.java:1074)
              at org.datanucleus.store.rdbms.scostore.JoinMapStore.putAll(JoinMapStore.java:224)
              at org.datanucleus.store.rdbms.mapping.java.MapMapping.postInsert(MapMapping.java:158)
              at org.datanucleus.store.rdbms.request.InsertRequest.execute(InsertRequest.java:522)
              at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.insertObjectInTable(RDBMSPersistenceHandler.java:162)
              at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.insertObject(RDBMSPersistenceHandler.java:138)
              at org.datanucleus.state.StateManagerImpl.internalMakePersistent(StateManagerImpl.java:3363)
              at org.datanucleus.state.StateManagerImpl.makePersistent(StateManagerImpl.java:3339)
              at org.datanucleus.ExecutionContextImpl.persistObjectInternal(ExecutionContextImpl.java:2080)
              at org.datanucleus.ExecutionContextImpl.persistObjectInternal(ExecutionContextImpl.java:2172)
              at org.datanucleus.store.rdbms.mapping.java.PersistableMapping.setObjectAsValue(PersistableMapping.java:603)
              at org.datanucleus.store.rdbms.mapping.java.PersistableMapping.setObject(PersistableMapping.java:357)
              at org.datanucleus.store.rdbms.fieldmanager.ParameterSetter.storeObjectField(ParameterSetter.java:191)
              at org.datanucleus.state.AbstractStateManager.providedObjectField(AbstractStateManager.java:1460)
              at org.datanucleus.state.StateManagerImpl.providedObjectField(StateManagerImpl.java:120)
              at org.apache.hadoop.hive.metastore.model.MStorageDescriptor.dnProvideField(MStorageDescriptor.java)
              at org.apache.hadoop.hive.metastore.model.MStorageDescriptor.dnProvideFields(MStorageDescriptor.java)
              at org.datanucleus.state.StateManagerImpl.provideFields(StateManagerImpl.java:1170)
              at org.datanucleus.store.rdbms.request.InsertRequest.execute(InsertRequest.java:292)
              at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.insertObjectInTable(RDBMSPersistenceHandler.java:162)
              at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.insertObject(RDBMSPersistenceHandler.java:138)
              at org.datanucleus.state.StateManagerImpl.internalMakePersistent(StateManagerImpl.java:3363)
              at org.datanucleus.state.StateManagerImpl.makePersistent(StateManagerImpl.java:3339)
              at org.datanucleus.ExecutionContextImpl.persistObjectInternal(ExecutionContextImpl.java:2080)
              at org.datanucleus.ExecutionContextImpl.persistObjectInternal(ExecutionContextImpl.java:2172)
              at org.datanucleus.store.rdbms.mapping.java.PersistableMapping.setObjectAsValue(PersistableMapping.java:603)
              at org.datanucleus.store.rdbms.mapping.java.PersistableMapping.setObject(PersistableMapping.java:357)
              at org.datanucleus.store.rdbms.fieldmanager.ParameterSetter.storeObjectField(ParameterSetter.java:191)
              at org.datanucleus.state.AbstractStateManager.providedObjectField(AbstractStateManager.java:1460)
              at org.datanucleus.state.StateManagerImpl.providedObjectField(StateManagerImpl.java:120)
              at org.apache.hadoop.hive.metastore.model.MTable.dnProvideField(MTable.java)
              at org.apache.hadoop.hive.metastore.model.MTable.dnProvideFields(MTable.java)
              at org.datanucleus.state.StateManagerImpl.provideFields(StateManagerImpl.java:1170)
              at org.datanucleus.store.rdbms.request.InsertRequest.execute(InsertRequest.java:292)
              at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.insertObjectInTable(RDBMSPersistenceHandler.java:162)
              at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.insertObject(RDBMSPersistenceHandler.java:138)
              at org.datanucleus.state.StateManagerImpl.internalMakePersistent(StateManagerImpl.java:3363)
              at org.datanucleus.state.StateManagerImpl.makePersistent(StateManagerImpl.java:3339)
              at org.datanucleus.ExecutionContextImpl.persistObjectInternal(ExecutionContextImpl.java:2080)
              at org.datanucleus.ExecutionContextImpl.persistObjectWork(ExecutionContextImpl.java:1923)
              at org.datanucleus.ExecutionContextImpl.persistObject(ExecutionContextImpl.java:1778)
              at org.datanucleus.ExecutionContextThreadedImpl.persistObject(ExecutionContextThreadedImpl.java:217)
              at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:724)
              at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:749)
              at org.apache.hadoop.hive.metastore.ObjectStore.createTable(ObjectStore.java:1464)
              at sun.reflect.GeneratedMethodAccessor73.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97)
              at com.sun.proxy.$Proxy27.createTable(Unknown Source)
              at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:2289)
              at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_req(HiveMetaStore.java:2495)
      Caused by: org.postgresql.util.PSQLException: ERROR: invalid byte sequence for encoding "UTF8": 0x00
              at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2532)
              at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2267)
              at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:312)
              at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:448)
              at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:369)
              at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:153)
              at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:119)
              at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61)
              at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java)
              at org.datanucleus.store.rdbms.ParamLoggingPreparedStatement.executeUpdate(ParamLoggingPreparedStatement.java:393)
              at org.datanucleus.store.rdbms.SQLController.executeStatementUpdate(SQLController.java:431)
              at org.datanucleus.store.rdbms.scostore.JoinMapStore.internalPut(JoinMapStore.java:1065)
      

        Attachments

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

            • Assignee:
              Unassigned Assign to me
              Reporter:
              Rajkumar Singh Rajkumar Singh

              Dates

              • Created:
                Updated:

                Issue deployment