Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-21761 Support table level replication in Hive
  3. HIVE-21762

REPL DUMP to support new format for replication policy input to take included tables list.

    XMLWordPrintableJSON

Details

    Description

      • REPL DUMP syntax:
        REPL DUMP <repl_policy> [FROM <last_repl_id> WITH <key_values_list>;
        
      • New format for the Replication policy have 3 parts all separated with Dot (.).
        1. First part is DB name.
        2. Second part is included list. Comma separated table names/regex with in square brackets[]. If square brackets are not there, then it is treated as single table replication which skips DB level events.
        3. Third part is excluded list. Comma separated table names/regex with in square brackets[].
        <db_name> -- Full DB replication which is currently supported
        <db_name>.['.*?']  -- Full DB replication
        <db_name>.[] -- Replicate just functions and not include any tables.
        <db_name>.['t1', 't2']  -- DB replication with static list of tables t1 and t2 included.
        <db_name>.['t1*', 't2', '*t3'].['t100', '5t3', 't4'] -- DB replication with all tables having prefix t1, with suffix t3 and include table t2 and exclude t100 which has the prefix t1, 5t3 which suffix t3 and t4.
        
      • Need to support regular expression of any format.
      • A table is included in dump only if it matches the regular expressions in included list and doesn't match the excluded list.

      Attachments

        1. HIVE-21762.01.patch
          45 kB
          Sankar Hariappan
        2. HIVE-21762.02.patch
          57 kB
          Sankar Hariappan
        3. HIVE-21762.03.patch
          62 kB
          Sankar Hariappan
        4. HIVE-21762.04.patch
          123 kB
          Sankar Hariappan
        5. HIVE-21762.05.patch
          143 kB
          Sankar Hariappan
        6. HIVE-21762.06.patch
          143 kB
          Sankar Hariappan
        7. HIVE-21762.07.patch
          165 kB
          Sankar Hariappan

        Issue Links

          Activity

            People

              sankarh Sankar Hariappan
              sankarh Sankar Hariappan
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 6h
                  6h