Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-13865

Changing char column of orc table to string/var char drops white space.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Not A Bug
    • 1.2.1
    • None
    • None
    • None

    Description

      Creating a orc table with char(16) column and insert some value with white space followed by characters:

      0: jdbc:hive2://os-r6-ifsmes-hiveserver2-11-4> create table test (c char(16)) stored as orc;
      No rows affected (0.1 seconds)
      0: jdbc:hive2://os-r6-ifsmes-hiveserver2-11-4> insert into table test values ('horton works     ');
      INFO  : Tez session hasn't been created yet. Opening session
      INFO  : Dag name: insert into table test values ('horton ...')(Stage-1)
      INFO  :
      
      INFO  : Status: Running (Executing on YARN cluster with App id application_1464222003837_0399)
      
      INFO  : Map 1: -/-
      INFO  : Map 1: 0/1
      INFO  : Map 1: 0/1
      INFO  : Map 1: 0/1
      INFO  : Map 1: 0(+1)/1
      INFO  : Map 1: 0(+1)/1
      INFO  : Map 1: 1/1
      INFO  : Loading data to table default.test from hdfs://os-r6-ifsmes-hiveserver2-11-5.openstacklocal:8020/apps/hive/warehouse/test/.hive-staging_hive_2016-05-26_17-43-07_098_2458959255563595485-1/-ext-10000
      INFO  : Table default.test stats: [numFiles=1, numRows=1, totalSize=267, rawDataSize=100]
      No rows affected (25.125 seconds)
      0: jdbc:hive2://os-r6-ifsmes-hiveserver2-11-4> select * from test;
      +-------------------+--+
      |      test.c       |
      +-------------------+--+
      | horton works      |
      +-------------------+--+
      1 row selected (0.077 seconds)
      0: jdbc:hive2://os-r6-ifsmes-hiveserver2-11-4> describe test;
      +-----------+------------+----------+--+
      | col_name  | data_type  | comment  |
      +-----------+------------+----------+--+
      | c         | char(16)   |          |
      +-----------+------------+----------+--+
      1 row selected (0.153 seconds)
      

      Then after changing the column to string, the white space is lost:

      0: jdbc:hive2://os-r6-ifsmes-hiveserver2-11-4> alter table test change column c c string;
      No rows affected (0.155 seconds)
      0: jdbc:hive2://os-r6-ifsmes-hiveserver2-11-4> describe test;
      +-----------+------------+----------+--+
      | col_name  | data_type  | comment  |
      +-----------+------------+----------+--+
      | c         | string     |          |
      +-----------+------------+----------+--+
      1 row selected (0.115 seconds)
      0: jdbc:hive2://os-r6-ifsmes-hiveserver2-11-4> select * from test;
      +---------------+--+
      |    test.c     |
      +---------------+--+
      | horton works  |
      +---------------+--+
      1 row selected (0.068 seconds)
      

      The issue is not seen with textfile formatted table:

      0: jdbc:hive2://os-r6-ifsmes-hiveserver2-11-4> create table test_text (c char(16)) stored as textfile;
      No rows affected (0.091 seconds)
      0: jdbc:hive2://os-r6-ifsmes-hiveserver2-11-4> insert into table test_text values ('horton works     ');
      INFO  : Session is already open
      INFO  : Dag name: insert into table test_text values ('ho...')(Stage-1)
      INFO  :
      
      INFO  : Status: Running (Executing on YARN cluster with App id application_1464222003837_0399)
      
      INFO  : Map 1: 0/1
      INFO  : Map 1: 0(+1)/1
      INFO  : Map 1: 1/1
      INFO  : Loading data to table default.test_text from hdfs://os-r6-ifsmes-hiveserver2-11-5.openstacklocal:8020/apps/hive/warehouse/test_text/.hive-staging_hive_2016-05-26_17-45-29_669_2888061873550824337-1/-ext-10000
      INFO  : Table default.test_text stats: [numFiles=1, numRows=1, totalSize=17, rawDataSize=16]
      No rows affected (6.849 seconds)
      0: jdbc:hive2://os-r6-ifsmes-hiveserver2-11-4> select * from test_text;
      +-------------------+--+
      |    test_text.c    |
      +-------------------+--+
      | horton works      |
      +-------------------+--+
      1 row selected (0.098 seconds)
      0: jdbc:hive2://os-r6-ifsmes-hiveserver2-11-4> describe test_text;
      +-----------+------------+----------+--+
      | col_name  | data_type  | comment  |
      +-----------+------------+----------+--+
      | c         | char(16)   |          |
      +-----------+------------+----------+--+
      1 row selected (0.127 seconds)
      0: jdbc:hive2://os-r6-ifsmes-hiveserver2-11-4> alter table test_text change column c c string;
      No rows affected (0.145 seconds)
      0: jdbc:hive2://os-r6-ifsmes-hiveserver2-11-4> describe test_text;
      +-----------+------------+----------+--+
      | col_name  | data_type  | comment  |
      +-----------+------------+----------+--+
      | c         | string     |          |
      +-----------+------------+----------+--+
      1 row selected (0.127 seconds)
      0: jdbc:hive2://os-r6-ifsmes-hiveserver2-11-4> select * from test_text;
      +-------------------+--+
      |    test_text.c    |
      +-------------------+--+
      | horton works      |
      +-------------------+--+
      1 row selected (0.066 seconds)
      

      Attachments

        Activity

          People

            mmccline Matt McCline
            taksaito Takahiko Saito
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: