Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-13697

ListBucketing feature does not support uppercase string.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Patch Available
    • Critical
    • Resolution: Unresolved
    • 1.2.1
    • None
    • Database/Schema
    • None
    • 1.2.1

    Description

      This is the feature:
      https://cwiki.apache.org/confluence/display/Hive/ListBucketing

      1. Good example:

      CREATE TABLE testskew (id INT, a STRING)
      SKEWED BY (a) ON ('abc', 'xyz') STORED AS DIRECTORIES;
      
      set hive.mapred.supports.subdirectories=true;
      set mapred.input.dir.recursive=true;
      
       INSERT OVERWRITE TABLE testskew 
       SELECT 123,'abc' FROM dual
       union all
       SELECT 123,'xyz' FROM dual
       union all
       SELECT 123,'others' FROM dual;
      
      # hadoop fs -ls /user/hive/warehouse/testskew
      Found 3 items
      drwxrwxrwx   - mapr mapr          1 2016-05-05 14:56
      /user/hive/warehouse/testskew/HIVE_DEFAULT_LIST_BUCKETING_DIR_NAME
      drwxrwxrwx   - mapr mapr          1 2016-05-05 14:56
      /user/hive/warehouse/testskew/a=abc
      drwxrwxrwx   - mapr mapr          1 2016-05-05 14:56
      /user/hive/warehouse/testskew/a=xyz
      

      This is good, because both "abc" and "xyz" directories got created.

      2. Bad example – This is the issue

      CREATE TABLE testskew2 (id INT, a STRING)
      SKEWED BY (a) ON ('aus', 'US') STORED AS DIRECTORIES;
      
      set hive.mapred.supports.subdirectories=true;
      set mapred.input.dir.recursive=true;
      
       INSERT OVERWRITE TABLE testskew2 
       SELECT 123, 'aus' FROM dual
       union all
       SELECT 123, 'US' FROM dual
       union all
       SELECT 123, 'others' FROM dual;
      

      You can see, only "aus" directory got created...

      # hadoop fs -ls /user/hive/warehouse/testskew2
      Found 2 items
      drwxrwxrwx   - mapr mapr          1 2016-05-05 15:11
      /user/hive/warehouse/testskew2/HIVE_DEFAULT_LIST_BUCKETING_DIR_NAME
      drwxrwxrwx   - mapr mapr          1 2016-05-05 15:11
      /user/hive/warehouse/testskew2/a=aus
      

      Attachments

        1. HIVE-13697.2.patch
          1 kB
          Oleksiy Sayankin
        2. HIVE-13697.1.patch
          1 kB
          Oleksiy Sayankin

        Issue Links

          Activity

            People

              osayankin Oleksiy Sayankin
              haozhu Hao Zhu
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated: