Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-21261

Incremental REPL LOAD adds redundant COPY and MOVE tasks for external table events.

Log workAgile BoardRank to TopRank to BottomVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      For external tables replication, the data gets copied as separate task based on data locations listed in _external_tables_info file in the dump. So, individual events such as ADD_PARTITION or INSERT on the external tables should avoid copying data. So, it is enough to create table/add partition DDL tasks. COPY and MOVE tasks should be skipped.

      Attachments

        1. HIVE-21261.01.patch
          22 kB
          Sankar Hariappan
        2. HIVE-21261.02.patch
          22 kB
          Sankar Hariappan

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            sankarh Sankar Hariappan Assign to me
            sankarh Sankar Hariappan
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - Not Specified
              Not Specified
              Remaining:
              Remaining Estimate - 0h
              0h
              Logged:
              Time Spent - 20m
              20m

              Issue deployment