[HADOOP-7199] Crash due to reuse of checksum files

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Not A Problem
• Affects Version/s: 0.20.2
• Component/s: fs

Environment

Cloudera CDH3B4 in pseudo mode on a Linux 2.6.32-28-generic #55-Ubuntu SMP x86_64 kernel, with Java HotSpot64-Bit Server VM (build 19.1-b02, mixed mode)

• Assignee: Unassigned
• Reporter: Lars Ailo Bongo
• Created: 18/Mar/11 16:08
• Updated: 27/Sep/21 10:37
• Resolved: 27/Sep/21 10:37

Description

copyFromLocalFile crashes if a cheksum file exists on the local filesystem and the checksum does not match the file content. This will for example crash "hadoop -fs put ./foo ./foo" with a non-descriptive error.

It is therefore not possible to do:

1. copyToLocalFile(hdfsFile, localFile) // creates checksum file
2. modify localFile
3. copyFromLocalFile(localFile, hdfsFile) // uses old checksum

Solution: do not reuse checksum files, or add a parameter to copyFromLocalFile that specifies that checksum files should not be reused.