[HIVE-28271] DirectSql fails for AlterPartitions

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.1.0
• Labels: hive-4.0.1-merged hive-4.0.1-must pull-request-available
• Assignee: Ayush Saxena
• Reporter: Ayush Saxena
• Created: 21/May/24 17:05
• Updated: 16/Jun/24 01:50
• Resolved: 22/May/24 17:12

Description

It fails at three places: (Misses Database Which Uses CLOB & Missing Boolean type conversions Checks

First:

2024-05-21T08:50:16,570  WARN [main] metastore.ObjectStore: Falling back to ORM path due to direct SQL failure (this is not an error): java.lang.ClassCastException: org.apache.derby.impl.jdbc.EmbedClob cannot be cast to java.lang.String at org.apache.hadoop.hive.metastore.ExceptionHandler.newMetaException(ExceptionHandler.java:152) at org.apache.hadoop.hive.metastore.Batchable.runBatched(Batchable.java:92) at org.apache.hadoop.hive.metastore.DirectSqlUpdatePart.getParams(DirectSqlUpdatePart.java:748) at org.apache.hadoop.hive.metastore.DirectSqlUpdatePart.updateParamTableInBatch(DirectSqlUpdatePart.java:715) at org.apache.hadoop.hive.metastore.DirectSqlUpdatePart.alterPartitions(DirectSqlUpdatePart.java:636) at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.alterPartitions(MetaStoreDirectSql.java:599) at org.apache.hadoop.hive.metastore.ObjectStore$20.getSqlResult(ObjectStore.java:5371);

Second:

2024-05-21T09:14:36,808  WARN [main] metastore.ObjectStore: Falling back to ORM path due to direct SQL failure (this is not an error): java.lang.ClassCastException: org.apache.derby.impl.jdbc.EmbedClob cannot be cast to java.lang.String at org.apache.hadoop.hive.metastore.ExceptionHandler.newMetaException(ExceptionHandler.java:152) at org.apache.hadoop.hive.metastore.Batchable.runBatched(Batchable.java:92) at org.apache.hadoop.hive.metastore.DirectSqlUpdatePart.updateCDInBatch(DirectSqlUpdatePart.java:1228) at org.apache.hadoop.hive.metastore.DirectSqlUpdatePart.updateStorageDescriptorInBatch(DirectSqlUpdatePart.java:888) at org.apache.hadoop.hive.metastore.DirectSqlUpdatePart.alterPartitions(DirectSqlUpdatePart.java:638) at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.alterPartitions(MetaStoreDirectSql.java:599) at org.apache.hadoop.hive.metastore.ObjectStore$20.getSqlResult(ObjectStore.java:5371);

Third: Missing Boolean check type

2024-05-21T09:35:44,063  WARN [main] metastore.ObjectStore: Falling back to ORM path due to direct SQL failure (this is not an error): java.sql.BatchUpdateException: A truncation error was encountered trying to shrink CHAR 'false' to length 1. at org.apache.hadoop.hive.metastore.ExceptionHandler.newMetaException(ExceptionHandler.java:152) at org.apache.hadoop.hive.metastore.Batchable.runBatched(Batchable.java:92) at org.apache.hadoop.hive.metastore.DirectSqlUpdatePart.lambda$updateSDInBatch$16(DirectSqlUpdatePart.java:926) at org.apache.hadoop.hive.metastore.DirectSqlUpdatePart.updateWithStatement(DirectSqlUpdatePart.java:656) at org.apache.hadoop.hive.metastore.DirectSqlUpdatePart.updateSDInBatch(DirectSqlUpdatePart.java:926) at org.apache.hadoop.hive.metastore.DirectSqlUpdatePart.updateStorageDescriptorInBatch(DirectSqlUpdatePart.java:900) at org.apache.hadoop.hive.metastore.DirectSqlUpdatePart.alterPartitions(DirectSqlUpdatePart.java:638) at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.alterPartitions(MetaStoreDirectSql.java:599) at org.apache.hadoop.hive.metastore.ObjectStore$20.getSqlResult(ObjectStore.java:5371);

Issue Links

• is caused by: HIVE-27530 Implement direct SQL for alter partitions to improve performance
• links to: GitHub Pull Request #5256