Hive
  1. Hive
  2. HIVE-936

dynamic partitions creation based on values

    Details

    • Type: Task Task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.7.0
    • Component/s: Query Processor
    • Labels:
      None

      Description

      If a Hive table is created as partitioned, DML could only inserted into one partitioin per query. Ideally partitions should be created on the fly based on the value of the partition columns. As an example:
      {{

      { create table T (a int, b string) partitioned by (ds string); insert overwrite table T select a, b, ds from S where ds >= '2009-11-01' and ds <= '2009-11-16'; }

      }}
      should be able to execute in one DML rather than possibley 16 DML for each distinct ds values. CTAS and alter table should be able to do the same thing:
      {{

      { create table T partitioned by (ds string) as select * from S where ds >= '2009-11-01' and ds <= '2009-11-16'; }

      }}
      and
      {{

      { create table T(a int, b string, ds string); insert overwrite table T select * from S where ds >= '2009-11-1' and ds <= '2009-11-16'; alter table T partitioned by (ds); }

      }}
      should all return the same results.

        Issue Links

          Activity

          Ning Zhang created issue -
          Namit Jain made changes -
          Field Original Value New Value
          Assignee Ning Zhang [ nzhang ]
          Namit Jain made changes -
          Link This issue is depended upon by HIVE-938 [ HIVE-938 ]
          Namit Jain made changes -
          Link This issue blocks HIVE-938 [ HIVE-938 ]
          Zheng Shao made changes -
          Link This issue is duplicated by HIVE-1002 [ HIVE-1002 ]
          Ning Zhang made changes -
          Attachment dp_design.txt [ 12437579 ]
          Ning Zhang made changes -
          Attachment dp_design.txt [ 12437909 ]
          Ning Zhang made changes -
          Attachment dp_design.txt [ 12437579 ]
          Paul Yang made changes -
          Link This issue is blocked by HIVE-1238 [ HIVE-1238 ]
          Ning Zhang made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Carl Steinbach made changes -
          Issue Type New Feature [ 2 ] Task [ 3 ]
          Carl Steinbach made changes -
          Component/s Query Processor [ 12312586 ]
          Carl Steinbach made changes -
          Fix Version/s 0.7.0 [ 12315150 ]
          Carl Steinbach made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Gavin made changes -
          Link This issue blocks HIVE-938 [ HIVE-938 ]
          Gavin made changes -
          Link This issue is depended upon by HIVE-938 [ HIVE-938 ]

            People

            • Assignee:
              Ning Zhang
              Reporter:
              Ning Zhang
            • Votes:
              8 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development