[MAPREDUCE-7226] option to not fail distcp when source file is being written to

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 3.2.0
• Component/s: distcp
• Assignee: Unassigned
• Reporter: Steven Rand
• Created: 23/Jul/19 06:54
• Updated: 24/Jul/19 22:22

Description

If a file is being written to during a distcp, then we'll throw an IOE because its size at the target will be less than its size at the source:

private void compareFileLengths(CopyListingFileStatus source, Path target,
	Configuration configuration, long targetLen)
	throws IOException {
	final Path sourcePath = source.getPath();
	FileSystem fs = sourcePath.getFileSystem(configuration);
	long srcLen = fs.getFileStatus(sourcePath).getLen();
	if (srcLen != targetLen)
	throw new IOException("Mismatch in length of source:" + sourcePath + " (" + srcLen +
	") and target:" + target + " (" + targetLen + ")");
	}

This happens even when the -i flag is given to distcp to ignore failures, since the only exceptions distcp will ignore are instances of CopyReadException, and compareFileLengths just throws an IOE.

This means that distcp can't be used for certain workflows. The particular one that I have in mind is incrementally copying data from a production cluster to a DR cluster. This can be handled nicely using distcp with the -update and -delete flags, but the problem is that clients might be modifying the production cluster while the distcp runs, thereby causing it to fail even when -i is given.

One idea is to:

1. Have compareFileLengths throw a custom exception. It doesn't make sense to use CopyReadException, since this isn't a failure on read, but we could make another subclass of IOE for writes.
2. Have the CopyMapper check for our new custom exception in the same way that it does for CopyReadException when the -i flag is given. Or, if we don't want to change the behavior of the existing flag, we could add a new flag.