Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5014

Extending DistCp through a custom CopyListing is not possible

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.23.0, 0.23.1, 0.23.3, 0.23.4, 0.23.5
    • 2.5.0
    • distcp
    • None

    Description

      • While it is possible to implement a custom CopyListing in DistCp, DistCp driver class doesn't allow for using this custom CopyListing.
      • Allow SimpleCopyListing to provide an option to exclude files (For instance it is useful to exclude FileOutputCommiter.SUCCEEDED_FILE_NAME during copy as premature copy can indicate that the entire data is available at the destination)

      Attachments

        1. MAPREDUCE-5014.patch
          8 kB
          Srikanth Sundarrajan
        2. MAPREDUCE-5014.patch
          13 kB
          Srikanth Sundarrajan
        3. m5014_20130322.patch
          16 kB
          Tsz-wo Sze
        4. m5014_20130322_b-1.patch
          17 kB
          Tsz-wo Sze
        5. m5014_20130322b_b-1.patch
          18 kB
          Tsz-wo Sze
        6. m5014_20130322b.patch
          17 kB
          Tsz-wo Sze

        Activity

          People

            sriksun Srikanth Sundarrajan
            sriksun Srikanth Sundarrajan
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - 24h
                24h
                Remaining:
                Remaining Estimate - 24h
                24h
                Logged:
                Time Spent - Not Specified
                Not Specified