Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-3746

Hive query fails if the table contains external partitions

    Details

      Description

      If Hive contains a table which has external partitions, Drill fails in partition pruning code, which causes the query to fail.

      CREATE TABLE external_partition_test (boolean_field BOOLEAN) PARTITIONED BY (boolean_part BOOLEAN);
      
      ALTER TABLE external_partition_test ADD PARTITION (boolean_part='true') LOCATION '/some/path';
      
      ALTER TABLE external_partition_test ADD PARTITION (boolean_part='false') LOCATION '/some/path';
      

      Query:

      SELECT * FROM hive.`default`.external_partition_test where boolean_part = false
      

      Exception:

      
      java.lang.StringIndexOutOfBoundsException
      
      String index out of range: -14
      
      at java.lang.String.substring(String.java:1875) ~[na:1.7.0_45]
      at org.apache.drill.exec.planner.sql.HivePartitionLocation.<init>(HivePartitionLocation.java:31) ~[classes/:na]
      at org.apache.drill.exec.planner.sql.HivePartitionDescriptor.getPartitions(HivePartitionDescriptor.java:117) ~[classes/:na]
      at org.apache.drill.exec.planner.logical.partition.PruneScanRule.doOnMatch(PruneScanRule.java:185) ~[classes/:na]
      at org.apache.drill.exec.planner.sql.logical.HivePushPartitionFilterIntoScan$2.onMatch(HivePushPartitionFilterIntoScan.java:92) ~[classes/:na]
      at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:228) ~[calcite-core-1.4.0-drill-r0.jar:1.4.0-drill-r0]
      

      Looking at HivePartitionLocation, it looks like we are depending on the organization of files on FileSystem to get the partition values. We should get the partition values from MetaStore.

        Attachments

          Activity

            People

            • Assignee:
              venki387 Venki Korukanti
              Reporter:
              vkorukanti Venki Korukanti
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: