[HADOOP-17042] Hadoop distcp throws "ERROR: Tools helper ///usr/lib/hadoop/libexec/tools/hadoop-distcp.sh was not found"

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.2.1, 3.1.3
• Fix Version/s: 3.2.2, 3.3.1, 3.4.0
• Component/s: tools/distcp
• Hadoop Flags: Reviewed
• Assignee: Aki Tanaka
• Reporter: Aki Tanaka
• Created: 14/May/20 17:41
• Updated: 22/Jul/21 05:01
• Resolved: 18/May/20 06:38

Description

On Hadoop 3.x, we see following "ERROR: Tools helper ///usr/lib/hadoop/libexec/tools/hadoop-distcp.sh was not found." message on the first line of the command output when running Hadoop DistCp.

$ hadoop distcp /path/to/src /user/hadoop/
ERROR: Tools helper ///usr/lib/hadoop/libexec/tools/hadoop-distcp.sh was not found.
2020-05-14 17:11:53,173 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=false, overwrite=false, append=false, useDiff=false, useRdiff=false, fromSnapshot=null, toSnapshot=null, skipCRC=false, blocking=true
..

This message was added by HADOOP-12857 and it would be an expected behavior.
DistCp calls 'hadoop_add_to_classpath_tools hadoop-distcp' when it starts, and the error is returned because the hadoop-distcp.sh does not exist in the tools directory.

However, that error message confuses us. Since this is not an user end configuration issue, I would think it's better to change the log level to debug (hadoop_debug).

Issue Links

• relates to: HADOOP-12857 Rework hadoop-tools