[HIVE-25020] Prepared statement related exceptions from Metastore when using org.mariadb.jdbc.Driver

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.3.6
• Assignee: Unassigned
• Reporter: Luca Toscano
• Created: 15/Apr/21 12:34
• Updated: 15/Apr/21 12:34

Description

Hi everybody,

I am running Apache Bigtop 1.5 on Debian, and I am currently in the process of migrating everything from Debian 9 (stretch) to 10 (buster). Part of the upgrade involves the libmysql-dev vs libmariadb-dev packages, since the former is not provided on Buster (hence the only jdbc driver available is org.mariadb.jdbc.Driver).

Bigtop 1.5 ships with hive 2.3.6, Buster ships with libmariadb-java 2.3.0.

When I upgraded the hive-site.xml file with javax.jdo.option.ConnectionDriverName=org.mariadb.jdbc.Driver I noticed some issues on the Metastore side, for example

2021-04-13T06:46:15,564 ERROR [pool-11-thread-144] metastore.RetryingHMSHandler: Retrying HMSHandler after 2000 ms (attempt 1 of 10) with error: javax.jdo.JDODataStoreException: Cannot set Object parameter: value = MANAGED_TABLE for column "" : Could not set parameter at position 1 (values was 'MANAGED_TABLE')
Query - conn:30368(M)  - "SELECT DISTINCT 'org.apache.hadoop.hive.metastore.model.MTable' AS NUCLEUS_TYPE,A0.CREATE_TIME,A0.LAST_ACCESS_TIME,A0.OWNER,A0.RETENTION,A0.IS_REWRITE_ENABLED,A0.TBL_NAME,A0.TBL_TYPE,A0.TBL_ID FROM TBLS A0 LEFT OUTER JOIN DBS B0 ON A0.DB_ID = B0.DB_ID WHERE LOWER(B0.`NAME`) LIKE 'default' ESCAPE '\\' AND ((((A0.TBL_TYPE = ?) OR (A0.TBL_TYPE = ?)) OR (A0.TBL_TYPE = ?)) OR (A0.TBL_TYPE = ?))"
        at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
        at org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:391)
        at org.datanucleus.api.jdo.JDOQuery.executeWithArray(JDOQuery.java:267)
        at org.apache.hadoop.hive.metastore.ObjectStore.getTableMeta(ObjectStore.java:1265)
        at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:101)
        at com.sun.proxy.$Proxy25.getTableMeta(Unknown Source)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_meta(HiveMetaStore.java:2006)
        at sun.reflect.GeneratedMethodAccessor74.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
        at com.sun.proxy.$Proxy26.get_table_meta(Unknown Source)
        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_meta.getResult(ThriftHiveMetastore.java:11429)
        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_meta.getResult(ThriftHiveMetastore.java:11413)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
        at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:594)
        at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:589)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1926)
        at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:589)
        at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

The above was originated from Hue while trying to fetch metadata about tables into a database. I tried to deploy libmysql-dev (stretch package deployed on buster), reverting javax.jdo.option.ConnectionDriverName to com.mysql.jdbc.Driver and all the issues disappeared. This is a workaround that can work as interim solution, but I'd like to know if what I encountered above is expected or if there is a suggested way to fix it (that could also be included in Hive 3.x, it will be packaged in the next Bigtop version so it would be perfectly acceptable for me).