[HDFS-11385] DistCp CopyCommitter should issue warning instead of throw exception when failed to delete file.

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Component/s: distcp
• Assignee: Yiqun Lin
• Reporter: Yongjun Zhang
• Created: 02/Feb/17 06:58
• Updated: 02/Feb/17 13:32

Description

CopyCommiter#deleteMissing(Configuration conf) does the following in a loop for all files that need to be deleted:

        if (result) {
          LOG.info("Deleted " + trgtFileStatus.getPath() + " - Missing at source");
          deletedEntries++;
        } else {
          throw new IOException("Unable to delete " + trgtFileStatus.getPath());
        }

If for some reason there is a failure to delete a file, then an exception is thrown and no other files will be deleted.

It seems more reasonable to issue a warning here instead of throw an exception.