[HIVE-21377] Using Oracle as HMS DB with DirectSQL

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.0, 3.0.0
• Fix Version/s: 4.0.0-alpha-1
• Component/s: Metastore
• Assignee: Rajkumar Singh
• Reporter: Bo
• Created: 04/Mar/19 09:47
• Updated: 25/Jan/23 15:19
• Resolved: 02/Apr/19 17:52

Description

When we use the Oracle as HMS DB, we saw this kind of contents in the HMS log accordingly:

2019-02-02 T08:23:57,102 WARN [Thread-12]: metastore.ObjectStore (ObjectStore.java:handleDirectSqlError(3741)) - Falling back to ORM path due to direct SQL failure (this is not an error): Cannot extract boolean from column value 0 at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.extractSqlBoolean(MetaStoreDirectSql.java:1031) at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.getPartitionsFromPartitionIds(MetaStoreDirectSql.java:728) at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.access$300(MetaStoreDirectSql.java:109) at org.apache.hadoop.hive.metastore.MetaStoreDirectSql$1.run(MetaStoreDirectSql.java:471) at org.apache.hadoop.hive.metastore.Batchable.runBatched(Batchable.java:73) at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.getPartitionsViaSqlFilter(MetaStoreDirectSql.java:462) at org.apache.hadoop.hive.metastore.ObjectStore$8.getSqlResult(ObjectStore.java:3392)

In Hive, we handle the Postgres, MySQL and Derby for the extractSqlBoolean.
But Oracle return the 0 or 1 for Boolean. So we need to modify the MetastoreDirectSqlUtils.java - [1]

So, could add this snip in this code?

  static Boolean extractSqlBoolean(Object value) throws MetaException {
    if (value == null) {
      return null;
    }
    if (value instanceof Boolean) {
      return (Boolean)value;
    }
    if (value instanceof Number) { // add
      try {
        return BooleanUtils.toBooleanObject((BigDecimal) value, 1, 0, null);
      } catch(IllegalArugmentExeception iae){
      // NOOP
      }
    if (value instanceof String) {
      try {
        return BooleanUtils.toBooleanObject((String) value, "Y", "N", null);
      } catch (IllegalArgumentException iae) {
        // NOOP
      }
    }
    throw new MetaException("Cannot extract boolean from column value " + value);
  }

 [1] -
https://github.com/apache/hive/blob/f51f108b761f0c88647f48f30447dae12b308f31/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetastoreDirectSqlUtils.java#L501-L527
 

Issue Links

• is duplicated by: HIVE-25077 Direct SQL to fetch column privileges in refreshPrivileges may be broken in postgres