Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-20882 Support Hive replication to a target cluster with hive.strict.managed.tables enabled.
  3. HIVE-21500

Disable conversion of managed table to external and vice versa at source via alter table.

    XMLWordPrintableJSON

Details

    Description

      Couple of scenarios for Hive2 to Hive3(strict managed tables enabled) replication where managed table is converted to external at source.
      Scenario-1: (ACID/MM table converted to external at target)
      1. Create non-ACID ORC format table.
      2. Insert some rows
      3. Replicate this create event which creates ACID table at target (due to migration rule). Each insert event adds transactional metadata in HMS corresponding to the current table.
      4. Convert table to external table using ALTER command at source.

      Scenario-2: (External table at target changes table location)
      1. Create non-ACID avro format table.
      2. Insert some rows
      3. Replicate this create event which creates external table at target (due to migration rule). The data path is chosen under default external warehouse directory.
      4. Convert table to external table using ALTER command at source.

      It is unable to convert an ACID table to external table at target. Also, it is hard to detect what would be the table type at target when perform this ALTER table operation at source.
      So, it is decided to disable conversion of managed table at source (Hive2) to EXTERNAL or vice-versa if the DB is enabled for replication and strict managed is disabled.

      Attachments

        1. HIVE-21500.01.patch
          14 kB
          Sankar Hariappan
        2. HIVE-21500.02.patch
          14 kB
          Sankar Hariappan
        3. HIVE-21500.03.patch
          15 kB
          Sankar Hariappan

        Issue Links

          Activity

            People

              sankarh Sankar Hariappan
              sankarh Sankar Hariappan
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 6.5h
                  6.5h