Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5014

Extending DistCp through a custom CopyListing is not possible

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0, 0.23.1, 0.23.3, 0.23.4, 0.23.5
    • Fix Version/s: 2.5.0
    • Component/s: distcp
    • Labels:
      None

      Description

      • While it is possible to implement a custom CopyListing in DistCp, DistCp driver class doesn't allow for using this custom CopyListing.
      • Allow SimpleCopyListing to provide an option to exclude files (For instance it is useful to exclude FileOutputCommiter.SUCCEEDED_FILE_NAME during copy as premature copy can indicate that the entire data is available at the destination)

        Attachments

        1. m5014_20130322b.patch
          17 kB
          Tsz Wo Nicholas Sze
        2. m5014_20130322b_b-1.patch
          18 kB
          Tsz Wo Nicholas Sze
        3. m5014_20130322_b-1.patch
          17 kB
          Tsz Wo Nicholas Sze
        4. m5014_20130322.patch
          16 kB
          Tsz Wo Nicholas Sze
        5. MAPREDUCE-5014.patch
          13 kB
          Srikanth Sundarrajan
        6. MAPREDUCE-5014.patch
          8 kB
          Srikanth Sundarrajan

          Activity

            People

            • Assignee:
              sriksun Srikanth Sundarrajan
              Reporter:
              sriksun Srikanth Sundarrajan
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 24h
                24h
                Remaining:
                Remaining Estimate - 24h
                24h
                Logged:
                Time Spent - Not Specified
                Not Specified