Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7312

Update DistCp v1 to optionally not use tmp location (branch-1 only)

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 1.2.1
    • 1.3.0
    • tools
    • None
    • Reviewed
    • Hide
      DistCp v1 currently copies files to a temporary location and then renames that to the specified destination. This can cause performance issues on file systems such as S3. A -skiptmp flag is added to bypass this step and copy directly to the destination.
      Show
      DistCp v1 currently copies files to a temporary location and then renames that to the specified destination. This can cause performance issues on file systems such as S3. A -skiptmp flag is added to bypass this step and copy directly to the destination.

    Description

      DistCp v1 currently copies files to a tmp location and then renames that to the specified destination. This can cause performance issues on filesystems such as S3. A -skiptmp flag will be added to bypass this step and copy directly to the destination. This feature mirrors a similar one added to HBase ExportSnapshot HBASE-11119

      NOTE: This is a branch-1 change only.

      Attachments

        1. HDFS-7312.008.patch
          32 kB
          Joseph Prosser
        2. HDFS-7312.007.patch
          37 kB
          Joseph Prosser
        3. HDFS-7312.006.patch
          21 kB
          Joseph Prosser
        4. HDFS-7312.005.patch
          20 kB
          Joseph Prosser
        5. HDFS-7312.004.patch
          115 kB
          Joseph Prosser
        6. HDFS-7312.003.patch
          16 kB
          Joseph Prosser
        7. HDFS-7312.002.patch
          0.9 kB
          Joseph Prosser
        8. HDFS-7312.001.patch
          16 kB
          Joseph Prosser
        9. HDFS-7312.patch
          16 kB
          Joseph Prosser

        Activity

          People

            jprosser Joseph Prosser
            jprosser Joseph Prosser
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - 72h
                72h
                Remaining:
                Remaining Estimate - 72h
                72h
                Logged:
                Time Spent - Not Specified
                Not Specified