Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-21731

Hive import fails, post upgrade of source 3.0 cluster, to a target 4.0 cluster with strict managed table set to true.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • None
    • None

    Description

      The scenario is 

      1. Replication policy is set with hive  3.0 source cluster (strict managed table set to false) and hive 4.0 target cluster with strict managed table set  true.
      2. User upgrades the 3.0 source cluster to 4.0 cluster using upgrade tool.
      3. The upgrade converts all managed tables to acid tables.
      4. In the next repl dump, user sets hive .repl .dump .include .acid .tables and hive .repl .bootstrap. acid. tables set true triggering bootstrap of newly converted ACID tables.
      5. As the old tables are non-txn tables, dump is not filtering the events even tough bootstrap acid table is set to true. This is causing the repl load to fail as the write id is not set in the table object.
      6. If we ignore the event replay, the bootstrap is failing with dump directory mismatch error.

      The fix should be 

      1. Ignore dumping the alter table event if bootstrap acid table is set true and the alter is converting a non-acid table to acid table.
      2. In case of bootstrap during incremental load, ignore the dump directory property set in table object.

      Attachments

        1. HIVE-21731.01.patch
          10 kB
          mahesh kumar behera
        2. HIVE-21731.02.patch
          13 kB
          mahesh kumar behera
        3. HIVE-21731.03.patch
          15 kB
          mahesh kumar behera
        4. HIVE-21731.04.patch
          13 kB
          mahesh kumar behera
        5. HIVE-21731.05.patch
          13 kB
          mahesh kumar behera

        Issue Links

          Activity

            People

              maheshk114 mahesh kumar behera
              maheshk114 mahesh kumar behera
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 3h 40m
                  3h 40m