[HIVE-23358] MSCK REPAIR should remove all insignificant zeroes from numeric partition values

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0-alpha-1
• Component/s: Standalone Metastore
• Labels: pull-request-available
• Target Version/s: 4.0.0
• Assignee: Adesh Kumar Rao
• Reporter: Adesh Kumar Rao
• Created: 03/May/20 04:59
• Updated: 17/Nov/22 08:51
• Resolved: 17/Aug/20 13:47

Description

For the following scenario
1. Have partitioned data path as follows.
hdfs://mycluster/datapath/t1/year=2020/month=03/day=10
hdfs://mycluster/datapath/t1/year=2020/month=03/day=11
2. create external table t1 (key int, value string) partitioned by (Year int, Month int, Day int) stored as orc location hdfs://mycluster/datapath/t1'';
3. msck repair table t1;
4. show partitions t1; 

+----------------------------+
|         partition          |
+----------------------------+
| year=2020/month=03/day=10  |
| year=2020/month=03/day=11 |
+----------------------------+

5.show table extended like 't1' partition (Year=2020, Month=03, Day=11);
will throw an error:

Error: Error while compiling statement: FAILED: SemanticException [Error 10006]: Partition not found {year=2020, month=3, day=11} (state=42000,code=10006)

When the partition directory are created without the extra zeroes, this works fine.

hdfs://mycluster/datapath/t1/year=2020/month=3/day=10
hdfs://mycluster/datapath/t1/year=2020/month=3/day=11

This happens because while searching for partitions, hive strips the extra "0" in month key and then queries the metastore (partSpec="year=2020/month=3/day=10") which returns no rows.

Issue Links

• is related to: HIVE-23347 MSCK REPAIR cannot discover partitions with upper case directory names.
• relates to: HIVE-10362 Support Type check/conversion in dynamic partition column
• links to: GitHub Pull Request #1050
• links to: review-request