Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7312

Update DistCp v1 to optionally not use tmp location (branch-1 only)

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.2.1
    • Fix Version/s: 1.3.0
    • Component/s: tools
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      DistCp v1 currently copies files to a temporary location and then renames that to the specified destination. This can cause performance issues on file systems such as S3. A -skiptmp flag is added to bypass this step and copy directly to the destination.
      Show
      DistCp v1 currently copies files to a temporary location and then renames that to the specified destination. This can cause performance issues on file systems such as S3. A -skiptmp flag is added to bypass this step and copy directly to the destination.

      Description

      DistCp v1 currently copies files to a tmp location and then renames that to the specified destination. This can cause performance issues on filesystems such as S3. A -skiptmp flag will be added to bypass this step and copy directly to the destination. This feature mirrors a similar one added to HBase ExportSnapshot HBASE-11119

      NOTE: This is a branch-1 change only.

        Attachments

        1. HDFS-7312.008.patch
          32 kB
          Joseph Prosser
        2. HDFS-7312.007.patch
          37 kB
          Joseph Prosser
        3. HDFS-7312.006.patch
          21 kB
          Joseph Prosser
        4. HDFS-7312.005.patch
          20 kB
          Joseph Prosser
        5. HDFS-7312.004.patch
          115 kB
          Joseph Prosser
        6. HDFS-7312.003.patch
          16 kB
          Joseph Prosser
        7. HDFS-7312.002.patch
          0.9 kB
          Joseph Prosser
        8. HDFS-7312.001.patch
          16 kB
          Joseph Prosser
        9. HDFS-7312.patch
          16 kB
          Joseph Prosser

          Activity

            People

            • Assignee:
              jprosser Joseph Prosser
              Reporter:
              jprosser Joseph Prosser
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 72h
                72h
                Remaining:
                Remaining Estimate - 72h
                72h
                Logged:
                Time Spent - Not Specified
                Not Specified