Hive
  1. Hive
  2. HIVE-6405 Support append feature for HCatalog
  3. HIVE-6406

Introduce immutable-table table property and if set, disallow insert-into

    Details

    • Release Note:
      Hide
      This patch introduces a new table property "immutable".

      If we create a table with TBLPROPERTIES("immutable"="true"), then INSERT INTO behaviour into that table will be disallowed if there is any data already present that the INSERT INTO would append to.

      INSERT INTO will still work if it is empty.

      INSERT OVERWRITE behaviour is not modified by this property, since an INSERT OVERWRITE's intent is effectively to drop and re-create.

      The desirability of having an immutable flag for a table is that it allows a table to be flagged to be protected against accidental updates due to a script loading data into it being run multiple times by mistake. With the flag set, the first insert would succeed, and successive inserts would fail, thus resulting in only one set of data in the table, instead of silently succeeding with 4 copies(say) of the data in the table.
      Show
      This patch introduces a new table property "immutable". If we create a table with TBLPROPERTIES("immutable"="true"), then INSERT INTO behaviour into that table will be disallowed if there is any data already present that the INSERT INTO would append to. INSERT INTO will still work if it is empty. INSERT OVERWRITE behaviour is not modified by this property, since an INSERT OVERWRITE's intent is effectively to drop and re-create. The desirability of having an immutable flag for a table is that it allows a table to be flagged to be protected against accidental updates due to a script loading data into it being run multiple times by mistake. With the flag set, the first insert would succeed, and successive inserts would fail, thus resulting in only one set of data in the table, instead of silently succeeding with 4 copies(say) of the data in the table.

      Description

      As part of HIVE-6405's attempt to make HCatalog and Hive behave in similar ways with regards to immutable tables, this is a companion task to introduce the notion of an immutable table, wherein all tables are not immutable by default, and have this be a table property. If this property is set for a table, and we attempt to write to a table that already has data (or a partition), disallow "INSERT INTO" into it from hive(if destination directory is non-empty). This property being set will allow hive to mimic HCatalog's current immutable-table property.

      1. HIVE-6406.3.patch
        21 kB
        Sushanth Sowmyan
      2. HIVE-6406.2.patch
        21 kB
        Sushanth Sowmyan
      3. HIVE-6406.patch
        22 kB
        Sushanth Sowmyan

        Issue Links

          Activity

          Sushanth Sowmyan created issue -
          Sushanth Sowmyan made changes -
          Field Original Value New Value
          Component/s HCatalog [ 12320409 ]
          Component/s Metastore [ 12312584 ]
          Component/s Query Processor [ 12312586 ]
          Component/s Thrift API [ 12314332 ]
          Sushanth Sowmyan made changes -
          Attachment HIVE-6406.patch [ 12628308 ]
          Sushanth Sowmyan made changes -
          Description As part of HIVE-6405's attempt to make HCatalog and Hive behave in similar ways with regards to immutable tables, this is a companion task to introduce the notion of an immutable table, wherein all tables are not immutable by default, and have this be a table property. If this property is set for a table, and we attempt to write to a table that already has data (or a partition), disallow "INSERT INTO" into it from hive. This property being set will allow hive to mimic HCatalog's current immutable-table property. As part of HIVE-6405's attempt to make HCatalog and Hive behave in similar ways with regards to immutable tables, this is a companion task to introduce the notion of an immutable table, wherein all tables are not immutable by default, and have this be a table property. If this property is set for a table, and we attempt to write to a table that already has data (or a partition), disallow "INSERT INTO" into it from hive(if destination directory is non-empty). This property being set will allow hive to mimic HCatalog's current immutable-table property.
          Sushanth Sowmyan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Sushanth Sowmyan made changes -
          Attachment HIVE-6406.2.patch [ 12628572 ]
          Sushanth Sowmyan made changes -
          Attachment HIVE-6406.3.patch [ 12629133 ]
          Sushanth Sowmyan made changes -
          Attachment HIVE-6406.3.patch [ 12629133 ]
          Sushanth Sowmyan made changes -
          Attachment HIVE-6406.3.patch [ 12629134 ]
          Ashutosh Chauhan made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 0.13.0 [ 12324986 ]
          Resolution Fixed [ 1 ]
          Sushanth Sowmyan made changes -
          Release Note This patch introduces a new table property "immutable".

          If we create a table with TBLPROPERTIES("immutable"="true"), then INSERT INTO behaviour into that table will be disallowed if there is any data already present that the INSERT INTO would append to.

          INSERT INTO will still work if it is empty.

          INSERT OVERWRITE behaviour is not modified by this property, since an INSERT OVERWRITE's intent is effectively to drop and re-create.

          The desirability of having an immutable flag for a table is that it allows a table to be flagged to be protected against accidental updates due to a script loading data into it being run multiple times by mistake. With the flag set, the first insert would succeed, and successive inserts would fail, thus resulting in only one set of data in the table, instead of silently succeeding with 4 copies(say) of the data in the table.
          Lefty Leverenz made changes -
          Link This issue is related to HIVE-6465 [ HIVE-6465 ]

            People

            • Assignee:
              Sushanth Sowmyan
              Reporter:
              Sushanth Sowmyan
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development