Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-21209

[Improvement] Exchange partitition to be metadata only change?

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.1.1
    • Fix Version/s: None
    • Component/s: Hive
    • Labels:
      None

      Description

      https://issues.apache.org/jira/browse/HIVE-14560
      Current implementation of the above jira is a metadata and a "copy" of the partition data on the DFS. Could possibly take a long time to copy the data for large partition data especially different storage clusters. When exchanging a partition from a HDFS to S3a or vice versa the data is copied and this is client copy operation and it can be very slow if the partition is very large.

      The customer would like the "exchange partition" operation to purely metadata. I would like to start a discussion on whether this improvement is to be made. Obviously, the current behavior will be supported but and option for it to be a metadata operation only needs to be evaluated.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                ngangam Naveen Gangam
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: