[HIVE-24433] AutoCompaction is not getting triggered for CamelCase Partition Values

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Labels: pull-request-available
• Assignee: Naresh P R
• Reporter: Naresh P R
• Created: 26/Nov/20 01:52
• Updated: 07/Dec/20 09:29
• Resolved: 07/Dec/20 09:29

Description

PartionKeyValue is getting converted into lowerCase in below 2 places.

https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L2728

https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L2851

Because of which TXN_COMPONENTS & HIVE_LOCKS tables are not having entries from proper partition values.

When query completes, the entry moves from TXN_COMPONENTS to COMPLETED_TXN_COMPONENTS. Hive AutoCompaction will not recognize the partition & considers it as invalid partition

create table abc(name string) partitioned by(city string) stored as orc tblproperties('transactional'='true');
insert into abc partition(city='Bangalore') values('aaa');

Example entry in COMPLETED_TXN_COMPONENTS

+-----------+--------------+--------------------+-------------------+---------------------+-------------+-------------------+
| CTC_TXNID | CTC_DATABASE | CTC_TABLE          | CTC_PARTITION     | CTC_TIMESTAMP       | CTC_WRITEID | CTC_UPDATE_DELETE |
+-----------+--------------+--------------------+-------------------+---------------------+-------------+-------------------+
|         2 | default      | abc    | city=bangalore    | 2020-11-25 09:26:59 |           1 | N                 |
+-----------+--------------+--------------------+-------------------+---------------------+-------------+-------------------+

 

AutoCompaction fails to get triggered with below error

2020-11-25T09:35:10,364 INFO [Thread-9]: compactor.Initiator (Initiator.java:run(98)) - Checking to see if we should compact default.abc.city=bangalore
2020-11-25T09:35:10,380 INFO [Thread-9]: compactor.Initiator (Initiator.java:run(155)) - Can't find partition default.compaction_test.city=bangalore, assuming it has been dropped and moving on

I verifed below 4 SQL's with my PR, those all produced correct PartitionKeyValue

i.e, COMPLETED_TXN_COMPONENTS.CTC_PARTITION="city=Bangalore"

insert into table abc PARTITION(CitY='Bangalore') values('Dan');
insert overwrite table abc partition(CiTy='Bangalore') select Name from abc;
update table abc set Name='xy' where CiTy='Bangalore';
delete from abc where CiTy='Bangalore';

Issue Links

• links to: GitHub Pull Request #1712