Hive
  1. Hive
  2. HIVE-5907

after adding new column to hive partition table,the new columns'value will be null be for reload all partitions

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Critical Critical
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      when add new columns on external partition table ,query new column will get NULL value

      0: jdbc:hive2://localhost:10000> create external table test_alex1_replacereloadpart (id string,myname string) partitioned by(dt string);
      No rows affected (0.395 seconds)
      0: jdbc:hive2://localhost:10000>  alter table test_alex1_replacereloadpart add partition (dt='2013-11-28') location '2013-11-28';
      No rows affected (0.339 seconds)
      0: jdbc:hive2://localhost:10000> insert overwrite table test_alex1_replacereloadpart partition(dt='2013-11-28') select id,myname from test_alex1;
      +-----+---------+
      | id  | myname  |
      +-----+---------+
      +-----+---------+
      No rows selected (24.073 seconds)
      0: jdbc:hive2://localhost:10000> select * from test_alex1_replacereloadpart;
      +-------------------------------------------------------------------+---------+-------------+
      |                                id                                 | myname  |     dt      |
      +-------------------------------------------------------------------+---------+-------------+
      | 20501B55B0C64273A5316C5078442569E425CC318ADC454CA163D72503489436  | 1       | 2013-11-28  |
      | 20501B55B0C64273A5316C5078442569E3D87D24CA4E42D9A200E391417A7994  | 1       | 2013-11-28  |
      | 20501B55B0C64273A5316C5078442569814086424B2942919C97DF1D2D2C1F55  | 1       | 2013-11-28  |
      | 205018A818FB43A38CC29FF701E47D8B205018A818FB43A38CC29FF701E47D8B  | 1       | 2013-11-28  |
      | 205016D04EDA435CB578D059F931221EB9F06E42815243A486B611B96366DD01  | 1       | 2013-11-28  |
      | 205016D04EDA435CB578D059F931221E205016D04EDA435CB578D059F931221E  | 1       | 2013-11-28  |
      | 20500D24895641DBBBDBE71640E57887EA06A0AEB4324117B8125FC815297B36  | 1       | 2013-11-28  |
      | 205006E8A3EB4CA1997D947D89C5FD1B205006E8A3EB4CA1997D947D89C5FD1B  | 1       | 2013-11-28  |
      | 2050054E19B44C4D992D97C1661A26C32050054E19B44C4D992D97C1661A26C3  | 1       | 2013-11-28  |
      | 204fa32a43ef4aefac3b391562c5a25b7149D57B47E74C6F8C22CC30292FADF1  | 1       | 2013-11-28  |
      +-------------------------------------------------------------------+---------+-------------+
      0: jdbc:hive2://localhost:10000> alter table test_alex1_replacereloadpart replace columns (id string,myname string,bee string);
      No rows affected (0.202 seconds)
      

      i delete old data from hive partition dir ,and add new data into the same dir

      0: jdbc:hive2://localhost:10000> select * from test_alex1_replacereloadpart;
      +-------------------------------------------------------------------+---------+------+-------------+
      |                                id                                 | myname  | bee  |     dt      |
      +-------------------------------------------------------------------+---------+------+-------------+
      | 20501B55B0C64273A5316C5078442569E425CC318ADC454CA163D72503489436  | 1       |      | 2013-11-28  |
      | 20501B55B0C64273A5316C5078442569E3D87D24CA4E42D9A200E391417A7994  | 1       |      | 2013-11-28  |
      | 20501B55B0C64273A5316C5078442569814086424B2942919C97DF1D2D2C1F55  | 1       |      | 2013-11-28  |
      | 205018A818FB43A38CC29FF701E47D8B205018A818FB43A38CC29FF701E47D8B  | 1       |      | 2013-11-28  |
      | 205016D04EDA435CB578D059F931221EB9F06E42815243A486B611B96366DD01  | 1       |      | 2013-11-28  |
      | 205016D04EDA435CB578D059F931221E205016D04EDA435CB578D059F931221E  | 1       |      | 2013-11-28  |
      | 20500D24895641DBBBDBE71640E57887EA06A0AEB4324117B8125FC815297B36  | 1       |      | 2013-11-28  |
      | 205006E8A3EB4CA1997D947D89C5FD1B205006E8A3EB4CA1997D947D89C5FD1B  | 1       |      | 2013-11-28  |
      | 2050054E19B44C4D992D97C1661A26C32050054E19B44C4D992D97C1661A26C3  | 1       |      | 2013-11-28  |
      | 204fa32a43ef4aefac3b391562c5a25b7149D57B47E74C6F8C22CC30292FADF1  | 1       |      | 2013-11-28  |
      +-------------------------------------------------------------------+---------+------+-------------+
      

      after reload the partition ,the new data appear

      0: jdbc:hive2://localhost:10000> alter table test_alex1_replacereloadpart drop partition (dt='2013-11-28');
      No rows affected (0.286 seconds)
      
      0: jdbc:hive2://localhost:10000> alter table test_alex1_replacereloadpart add partition (dt='2013-11-28') location '2013-11-28';
      No rows affected (0.551 seconds)
      0: jdbc:hive2://localhost:10000> select * from test_alex1_replacereloadpart;
      +-------------------------------------------------------------------+---------+------+-------------+
      |                                id                                 | myname  | bee  |     dt      |
      +-------------------------------------------------------------------+---------+------+-------------+
      | 20501B55B0C64273A5316C5078442569E425CC318ADC454CA163D72503489436  | 1       | 10   | 2013-11-28  |
      | 20501B55B0C64273A5316C5078442569E3D87D24CA4E42D9A200E391417A7994  | 1       | 9    | 2013-11-28  |
      | 20501B55B0C64273A5316C5078442569814086424B2942919C97DF1D2D2C1F55  | 1       | 8    | 2013-11-28  |
      | 205018A818FB43A38CC29FF701E47D8B205018A818FB43A38CC29FF701E47D8B  | 1       | 7    | 2013-11-28  |
      | 205016D04EDA435CB578D059F931221EB9F06E42815243A486B611B96366DD01  | 1       | 6    | 2013-11-28  |
      | 205016D04EDA435CB578D059F931221E205016D04EDA435CB578D059F931221E  | 1       | 5    | 2013-11-28  |
      | 20500D24895641DBBBDBE71640E57887EA06A0AEB4324117B8125FC815297B36  | 1       | 4    | 2013-11-28  |
      | 205006E8A3EB4CA1997D947D89C5FD1B205006E8A3EB4CA1997D947D89C5FD1B  | 1       | 3    | 2013-11-28  |
      | 2050054E19B44C4D992D97C1661A26C32050054E19B44C4D992D97C1661A26C3  | 1       | 2    | 2013-11-28  |
      | 204fa32a43ef4aefac3b391562c5a25b7149D57B47E74C6F8C22CC30292FADF1  | 1       | 1    | 2013-11-28  |
      +-------------------------------------------------------------------+---------+------+-------------+
      

        Activity

        There are no comments yet on this issue.

          People

          • Assignee:
            Unassigned
            Reporter:
            alex kim
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:

              Development