Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.7.0
    • Component/s: Metastore, Query Processor
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      There should be a way to bring a table/partition offline.
      At that time, no read/write operations should be supported on that table.

      It would be very useful for housekeeping operations

      1. HIVE-1413.1.patch
        169 kB
        Siying Dong
      2. HIVE-1413.2.patch
        105 kB
        Siying Dong
      3. HIVE-1413.3.patch
        113 kB
        Siying Dong
      4. HIVE-1413.4.patch
        120 kB
        Siying Dong

        Issue Links

          Activity

          Namit Jain created issue -
          Namit Jain made changes -
          Field Original Value New Value
          Assignee Paul Yang [ pauly ] Siying Dong [ sdong ]
          John Sichi made changes -
          Fix Version/s 0.7.0 [ 12315150 ]
          Fix Version/s 0.6.0 [ 12314524 ]
          Hide
          Siying Dong added a comment -

          It's not a check-in-ready patch. It is tested locally but wasn't tested in cluster yet. Since it is involved in metastore change, want to do a preliminary first to make sure the metastore change is OK before testing on clusters.

          (1) modify thrift model and jdo to add extra columns "protectmode"
          (2) modify respective metastore codes to fetch and set the new columns
          (3) add command "alter table xxx set protectmode ..." and "alter table xxx alter partition (...) set protectmode ..."
          (4) When parsing and generating test, make sure we check protect mode of all tables/partitions read or written. Throw error if that is the case.
          (5) unit tests for it.

          Show
          Siying Dong added a comment - It's not a check-in-ready patch. It is tested locally but wasn't tested in cluster yet. Since it is involved in metastore change, want to do a preliminary first to make sure the metastore change is OK before testing on clusters. (1) modify thrift model and jdo to add extra columns "protectmode" (2) modify respective metastore codes to fetch and set the new columns (3) add command "alter table xxx set protectmode ..." and "alter table xxx alter partition (...) set protectmode ..." (4) When parsing and generating test, make sure we check protect mode of all tables/partitions read or written. Throw error if that is the case. (5) unit tests for it.
          Siying Dong made changes -
          Attachment HIVE-1414.1.patch [ 12450350 ]
          Siying Dong made changes -
          Attachment HIVE-1414.1.patch [ 12450350 ]
          Siying Dong made changes -
          Attachment HIVE-1414.1.patch [ 12450352 ]
          Siying Dong made changes -
          Attachment HIVE-1414.1.patch [ 12450352 ]
          Siying Dong made changes -
          Attachment HIVE-1414.1.patch [ 12450353 ]
          Hide
          John Sichi added a comment -

          Siying, you named the patch file "HIVE-1414.1.patch", but this is HIVE-1413.

          Show
          John Sichi added a comment - Siying, you named the patch file " HIVE-1414 .1.patch", but this is HIVE-1413 .
          Hide
          John Sichi added a comment -

          Metastore upgrade scripts will need to be added in HIVE-1427.

          Show
          John Sichi added a comment - Metastore upgrade scripts will need to be added in HIVE-1427 .
          Hide
          John Sichi added a comment -

          For the upgrade script, we'll need to specify DEFAULT on ADD COLUMN so existing tables remain ONLINE by default.

          Show
          John Sichi added a comment - For the upgrade script, we'll need to specify DEFAULT on ADD COLUMN so existing tables remain ONLINE by default.
          Hide
          Siying Dong added a comment -

          Made some modifications: make column typed string and others...

          Show
          Siying Dong added a comment - Made some modifications: make column typed string and others...
          Siying Dong made changes -
          Attachment HIVE-1413.1.patch [ 12450653 ]
          Siying Dong made changes -
          Attachment HIVE-1414.1.patch [ 12450353 ]
          Hide
          Siying Dong added a comment -

          1. use parameters without making any change to metastore
          2. add support of no_drop mode. Make every mode like bitmap instead of levels
          3. cover the case of dropping tables

          Show
          Siying Dong added a comment - 1. use parameters without making any change to metastore 2. add support of no_drop mode. Make every mode like bitmap instead of levels 3. cover the case of dropping tables
          Siying Dong made changes -
          Attachment HIVE-1413.2.patch [ 12451200 ]
          Hide
          Siying Dong added a comment -

          use more consolidated approach to try to check it in validate() instead of everywhere.
          change syntax a little bit.
          Add some more unit test cases.

          Show
          Siying Dong added a comment - use more consolidated approach to try to check it in validate() instead of everywhere. change syntax a little bit. Add some more unit test cases.
          Siying Dong made changes -
          Attachment HIVE-1413.3.patch [ 12451288 ]
          Siying Dong made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Namit Jain added a comment -

          +1

          will commit if the tests pass

          Show
          Namit Jain added a comment - +1 will commit if the tests pass
          Hide
          Siying Dong added a comment -

          The previous one missed two test output files...

          Show
          Siying Dong added a comment - The previous one missed two test output files...
          Siying Dong made changes -
          Attachment HIVE-1413.4.patch [ 12451298 ]
          Hide
          Namit Jain added a comment -

          committed. Thanks Siying

          Show
          Namit Jain added a comment - committed. Thanks Siying
          Namit Jain made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Resolution Fixed [ 1 ]
          Carl Steinbach made changes -
          Link This issue requires HIVE-1876 [ HIVE-1876 ]
          Carl Steinbach made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Hide
          Lefty Leverenz added a comment -

          Documented in the wiki here:

          Show
          Lefty Leverenz added a comment - Documented in the wiki here: DDL – Alter Table/Partition Protections
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          48d 4h 59m 1 Siying Dong 05/Aug/10 01:20
          Patch Available Patch Available Resolved Resolved
          4h 18m 1 Namit Jain 05/Aug/10 05:39
          Resolved Resolved Closed Closed
          498d 18h 22m 1 Carl Steinbach 17/Dec/11 00:01

            People

            • Assignee:
              Siying Dong
              Reporter:
              Namit Jain
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development