Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-16900

optimization to give distcp a list of input files to copy to a destination target directory during repl load

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Duplicate
    • 3.0.0
    • 3.0.0
    • HiveServer2
    • None

    Description

      During repl Copy currently we only allow operations per file as against list of files supported by distcp, During bootstrap table/partitions load it will be great to load all files listed in 

      _files

      in a single distcp job to make it more efficient, this would require changes to the shims sub project in hive to additionally expose api's which take multiple source files.

      Attachments

        Issue Links

          is cloned by

          Sub-task - The sub-task of the issue HIVE-16901 Distcp optimization - One distcp per ReplCopyTask

          • Major - Major loss of function.
          • Closed

          Activity

            People

              anishek Anishek Agarwal
              anishek Anishek Agarwal
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: