Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-14560

Support exchange partition between s3 and hdfs tables

Log workAgile BoardRank to TopRank to BottomVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0
    • Component/s: None
    • Labels:
      None

      Description

      alter table s3_tbl exchange partition (country='USA', state='CA') with table hdfs_tbl;
      

      results in:

      Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception: java.lang.IllegalArgumentException Wrong FS: s3a://hive-on-s3/s3_tbl/country=USA/state=CA, expected: hdfs://localhost:9000) (state=08S01,code=1)
      

      because the check for whether the s3 destination table path exists occurs on the hdfs filesystem.

      Furthermore, exchanging between s3 to hdfs fails because the hdfs rename operation is not supported across filesystems. Fix uses copy + deletion in the case that the file systems differ.

        Attachments

        1. HIVE-14560.02.patch
          4 kB
          Abdullah Yousufi
        2. HIVE-14560.patch
          5 kB
          Abdullah Yousufi

        Issue Links

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

            • Assignee:
              ayousufi Abdullah Yousufi Assign to me
              Reporter:
              ayousufi Abdullah Yousufi

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment