Hive
  1. Hive
  2. HIVE-7441

Custom partition scheme gets rewritten with hive scheme upon concatenate

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.10.0, 0.11.0, 0.12.0
    • Fix Version/s: 0.14.0
    • Component/s: CLI
    • Labels:
      None
    • Environment:

      CDH4.5 and CDH5.0

      Description

      If I take a given data directories. The directories contain a data file that is rc format and only contains one character "1".

      /j1/part1
      /j1/part2
      

      Create the table over the directories using the following command:

      create table j1 (a int) partitioned by (b string) stored as rcfile location '/j1' ;
      

      I add these directories to a table for example j1 using the following commands:

      alter table j1 add partition (b = 'part1') location '/j1/part1';
      alter table j1 add partition (b = 'part2') location '/j1/part2';
      

      I then do the following command to the first partition:

      alter table j1 partition (b = 'part1') concatenate;
      

      Hive changes the partition location from on hdfs

      /j1/part1
      

      to

      /j1/b=part1
      

      However it does not update the partition location in the metastore and partition is then lost to the table. It is hard to find this out until you start querying your data and notice there is missing data. The table even still shows the partition when you do "show partitions".

      1. HIVE-7441.patch
        10 kB
        Chaoyu Tang
      2. HIVE-7441.1.patch
        10 kB
        Chaoyu Tang

        Activity

        Thejas M Nair made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Szehon Ho made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Chaoyu Tang made changes -
        Attachment HIVE-7441.1.patch [ 12659967 ]
        Chaoyu Tang made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Fix Version/s 0.14.0 [ 12326450 ]
        Chaoyu Tang made changes -
        Attachment HIVE-7441.patch [ 12659786 ]
        Chaoyu Tang made changes -
        Field Original Value New Value
        Assignee Chaoyu Tang [ ctang.ma ]
        Johndee Burks created issue -

          People

          • Assignee:
            Chaoyu Tang
            Reporter:
            Johndee Burks
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development