Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14788

Use dynamic regex filter to ignore copy of source files in Distcp

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Done
    • 3.2.1
    • 3.3.0
    • distcp
    • None

    Description

      There is a feature in Distcp where we can ignore specific files to get copied to the destination. This is currently based on a filter regex which is read from a specific file. The process of creating different regex file for different distcp jobs seems like a tedious task. What we are proposing is to expose a regex_filter parameter which can be set during Distcp job creation and use this filter in a new implementation CopyFilter class. 

      Attachments

        Issue Links

          Activity

            People

              mukund-thakur Mukund Thakur
              mukund-thakur Mukund Thakur
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: