[HADOOP-16049] DistCp result has data and checksum mismatch when blocks per chunk > 0

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.9.2
• Fix Version/s: 2.9.3
• Component/s: tools/distcp
• Assignee: Kai Xie
• Reporter: Kai Xie
• Created: 17/Jan/19 14:17
• Updated: 03/Mar/19 07:56
• Resolved: 27/Jan/19 16:58

Description

In 2.9.2 RetriableFileCopyCommand.copyBytes,

int bytesRead = readBytes(inStream, buf, sourceOffset);
while (bytesRead >= 0) {
  ...
  if (action == FileAction.APPEND) {
    sourceOffset += bytesRead;
  }
  ... // write to dst
  bytesRead = readBytes(inStream, buf, sourceOffset);
}

it does a positioned read but the position (`sourceOffset` here) is never updated when blocks per chunk is set to > 0 (which always disables append action). So for chunk with offset != 0, it will keep copying the first few bytes again and again, causing result to have data & checksum mismatch.

To re-produce this issue, in branch-2, update BLOCK_SIZE to 10240 (> default copy buffer size) in class TestDistCpSystem and run it.

HADOOP-15292 has resolved the issue reported in this ticket in trunk/branch-3.1/branch-3.2 by not using the positioned read, but has not been backported to branch-2 yet

 

 

Issue Links

• is related to: HADOOP-11794 Enable distcp to copy blocks in parallel
• relates to: HADOOP-16158 DistCp to support checksum validation when copy blocks in parallel
• requires: HADOOP-15292 Distcp's use of pread is slowing it down.