Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-25020

Prepared statement related exceptions from Metastore when using org.mariadb.jdbc.Driver

Log workAgile BoardRank to TopRank to BottomAdd voteVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.3.6
    • None
    • None
    • None

    Description

      Hi everybody,

      I am running Apache Bigtop 1.5 on Debian, and I am currently in the process of migrating everything from Debian 9 (stretch) to 10 (buster). Part of the upgrade involves the libmysql-dev vs libmariadb-dev packages, since the former is not provided on Buster (hence the only jdbc driver available is org.mariadb.jdbc.Driver).

      Bigtop 1.5 ships with hive 2.3.6, Buster ships with libmariadb-java 2.3.0.

      When I upgraded the hive-site.xml file with javax.jdo.option.ConnectionDriverName=org.mariadb.jdbc.Driver I noticed some issues on the Metastore side, for example

      2021-04-13T06:46:15,564 ERROR [pool-11-thread-144] metastore.RetryingHMSHandler: Retrying HMSHandler after 2000 ms (attempt 1 of 10) with error: javax.jdo.JDODataStoreException: Cannot set Object parameter: value = MANAGED_TABLE for column "" : Could not set parameter at position 1 (values was 'MANAGED_TABLE')
      Query - conn:30368(M)  - "SELECT DISTINCT 'org.apache.hadoop.hive.metastore.model.MTable' AS NUCLEUS_TYPE,A0.CREATE_TIME,A0.LAST_ACCESS_TIME,A0.OWNER,A0.RETENTION,A0.IS_REWRITE_ENABLED,A0.TBL_NAME,A0.TBL_TYPE,A0.TBL_ID FROM TBLS A0 LEFT OUTER JOIN DBS B0 ON A0.DB_ID = B0.DB_ID WHERE LOWER(B0.`NAME`) LIKE 'default' ESCAPE '\\' AND ((((A0.TBL_TYPE = ?) OR (A0.TBL_TYPE = ?)) OR (A0.TBL_TYPE = ?)) OR (A0.TBL_TYPE = ?))"
              at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
              at org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:391)
              at org.datanucleus.api.jdo.JDOQuery.executeWithArray(JDOQuery.java:267)
              at org.apache.hadoop.hive.metastore.ObjectStore.getTableMeta(ObjectStore.java:1265)
              at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:101)
              at com.sun.proxy.$Proxy25.getTableMeta(Unknown Source)
              at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_meta(HiveMetaStore.java:2006)
              at sun.reflect.GeneratedMethodAccessor74.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
              at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
              at com.sun.proxy.$Proxy26.get_table_meta(Unknown Source)
              at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_meta.getResult(ThriftHiveMetastore.java:11429)
              at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_meta.getResult(ThriftHiveMetastore.java:11413)
              at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
              at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
              at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:594)
              at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:589)
              at java.security.AccessController.doPrivileged(Native Method)
              at javax.security.auth.Subject.doAs(Subject.java:422)
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1926)
              at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:589)
              at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
      

      The above was originated from Hue while trying to fetch metadata about tables into a database. I tried to deploy libmysql-dev (stretch package deployed on buster), reverting javax.jdo.option.ConnectionDriverName to com.mysql.jdbc.Driver and all the issues disappeared. This is a workaround that can work as interim solution, but I'd like to know if what I encountered above is expected or if there is a suggested way to fix it (that could also be included in Hive 3.x, it will be packaged in the next Bigtop version so it would be perfectly acceptable for me).

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned Assign to me
            elukey Luca Toscano

            Dates

              Created:
              Updated:

              Issue deployment