[DRILL-3746] Hive query fails if the table contains external partitions

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.2.0
• Component/s: Query Planning & Optimization
• Labels: no_verified_test
• Assignee: Venki Korukanti
• Reporter: Venki Korukanti
• Created: 08/Sep/15 14:19
• Updated: 29/Sep/15 16:58
• Resolved: 10/Sep/15 17:18

Description

If Hive contains a table which has external partitions, Drill fails in partition pruning code, which causes the query to fail.

CREATE TABLE external_partition_test (boolean_field BOOLEAN) PARTITIONED BY (boolean_part BOOLEAN);

ALTER TABLE external_partition_test ADD PARTITION (boolean_part='true') LOCATION '/some/path';

ALTER TABLE external_partition_test ADD PARTITION (boolean_part='false') LOCATION '/some/path';

Query:

SELECT * FROM hive.`default`.external_partition_test where boolean_part = false

Exception:

java.lang.StringIndexOutOfBoundsException

String index out of range: -14

at java.lang.String.substring(String.java:1875) ~[na:1.7.0_45]
at org.apache.drill.exec.planner.sql.HivePartitionLocation.<init>(HivePartitionLocation.java:31) ~[classes/:na]
at org.apache.drill.exec.planner.sql.HivePartitionDescriptor.getPartitions(HivePartitionDescriptor.java:117) ~[classes/:na]
at org.apache.drill.exec.planner.logical.partition.PruneScanRule.doOnMatch(PruneScanRule.java:185) ~[classes/:na]
at org.apache.drill.exec.planner.sql.logical.HivePushPartitionFilterIntoScan$2.onMatch(HivePushPartitionFilterIntoScan.java:92) ~[classes/:na]
at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:228) ~[calcite-core-1.4.0-drill-r0.jar:1.4.0-drill-r0]

Looking at HivePartitionLocation, it looks like we are depending on the organization of files on FileSystem to get the partition values. We should get the partition values from MetaStore.