Hive
  1. Hive
  2. HIVE-3576

Regression: ALTER TABLE DROP IF EXISTS PARTITION throws a SemanticException if Partition is not found

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Cannot Reproduce
    • Affects Version/s: 0.9.0
    • Fix Version/s: 0.11.0
    • Component/s: Metastore, Query Processor
    • Labels:
      None

      Description

      Doing a simple "ALTER TABLE testtable DROP IF EXISTS PARTITION(dt=NONEXISTENTPARTITION)" fails with a SemanticException of the 10006 kind (INVALID_PARTITION).

      This does not respect the hive.exec.drop.ignorenonexistent condition either, since there are no if-check-wraps around this area, when fetching partitions from the store.

        Activity

        Hide
        Harsh J added a comment -

        Sorry, let me explain with an example:

        On 0.7.1:

        hive> create table test(a string) partitioned by (dt string);
        OK
        Time taken: 21.049 seconds
        hive> alter table test drop if exists partition(dt=2007);
        OK
        Time taken: 1.297 seconds
        

        On 0.9.0:

        hive> create table test(a string) partitioned by (dt string);
        OK
        Time taken: 2.133 seconds
        hive> alter table test drop if exists partition(dt=2007);
        FAILED: SemanticException [Error 10006]: Partition not found dt = 2007
        

        Again, if we make sure to use 'strings', on 0.9.0, it works:

        hive> alter table test drop if exists partition(dt='2007');
        OK
        Time taken: 0.684 seconds
        
        Show
        Harsh J added a comment - Sorry, let me explain with an example: On 0.7.1: hive> create table test(a string) partitioned by (dt string); OK Time taken: 21.049 seconds hive> alter table test drop if exists partition(dt=2007); OK Time taken: 1.297 seconds On 0.9.0: hive> create table test(a string) partitioned by (dt string); OK Time taken: 2.133 seconds hive> alter table test drop if exists partition(dt=2007); FAILED: SemanticException [Error 10006]: Partition not found dt = 2007 Again, if we make sure to use 'strings', on 0.9.0, it works: hive> alter table test drop if exists partition(dt='2007'); OK Time taken: 0.684 seconds
        Hide
        Harsh J added a comment -

        This also only happens on tables partitioned by a string type key, not int types, for example:

        hive> create table str_part_table (a string) partitioned by (dt string);
        OK
        Time taken: 0.054 seconds
        hive> create table int_part_table (a string) partitioned by (dt int);   
        OK
        Time taken: 0.031 seconds
        hive> alter table str_part_table drop if exists partition (dt=2007);
        FAILED: SemanticException [Error 10006]: Partition not found dt = 2007
        hive> alter table int_part_table drop if exists partition (dt=2007);
        OK
        Time taken: 0.091 seconds
        hive> alter table str_part_table drop if exists partition (dt='2007');
        OK
        Time taken: 0.06 seconds
        hive> alter table int_part_table drop if exists partition (dt='2007');
        OK
        Time taken: 0.065 seconds
        
        Show
        Harsh J added a comment - This also only happens on tables partitioned by a string type key, not int types, for example: hive> create table str_part_table (a string) partitioned by (dt string); OK Time taken: 0.054 seconds hive> create table int_part_table (a string) partitioned by (dt int ); OK Time taken: 0.031 seconds hive> alter table str_part_table drop if exists partition (dt=2007); FAILED: SemanticException [Error 10006]: Partition not found dt = 2007 hive> alter table int_part_table drop if exists partition (dt=2007); OK Time taken: 0.091 seconds hive> alter table str_part_table drop if exists partition (dt='2007'); OK Time taken: 0.06 seconds hive> alter table int_part_table drop if exists partition (dt='2007'); OK Time taken: 0.065 seconds
        Hide
        Kanwaljit Singh added a comment -

        Any updates on this issue?

        Show
        Kanwaljit Singh added a comment - Any updates on this issue?
        Hide
        Harsh J added a comment -

        Unable to reproduce in 0.11+. Resolving.

        hive> create table str_part_table (a string) partitioned by (dt string);
        OK
        Time taken: 16.108 seconds
        hive> create table int_part_table (a string) partitioned by (dt int);
        OK
        Time taken: 0.705 seconds
        hive> alter table str_part_table drop if exists partition (dt=2007);
        OK
        Time taken: 1.38 seconds
        hive> alter table int_part_table drop if exists partition (dt=2007);
        OK
        Time taken: 3.494 seconds
        hive> alter table str_part_table drop if exists partition (dt='2007');
        OK
        Time taken: 0.091 seconds
        hive> alter table int_part_table drop if exists partition (dt='2007');
        OK
        Time taken: 0.101 seconds
        
        Show
        Harsh J added a comment - Unable to reproduce in 0.11+. Resolving. hive> create table str_part_table (a string) partitioned by (dt string); OK Time taken: 16.108 seconds hive> create table int_part_table (a string) partitioned by (dt int ); OK Time taken: 0.705 seconds hive> alter table str_part_table drop if exists partition (dt=2007); OK Time taken: 1.38 seconds hive> alter table int_part_table drop if exists partition (dt=2007); OK Time taken: 3.494 seconds hive> alter table str_part_table drop if exists partition (dt='2007'); OK Time taken: 0.091 seconds hive> alter table int_part_table drop if exists partition (dt='2007'); OK Time taken: 0.101 seconds

          People

          • Assignee:
            Unassigned
            Reporter:
            Harsh J
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development