[BIGTOP-3735] Sqoop bin script doesn't work on Bigtop-3.1.0

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.2.0, 3.1.1

Environment

Bigtop version: 3.1.0
System version: centos7

• Assignee: wuzhiguo
• Reporter: Yu Hou
• Created: 12/Jul/22 12:27
• Updated: 14/Jul/22 03:30
• Resolved: 14/Jul/22 03:30

Description

Bigtop version: 3.1.0
System version: centos7
Someone could take a look at it for me, I encountered the following problems:

/usr/lib/sqoop/bin/sqoop import \> --connect jdbc:mysql://bdp81:3306/ambari \> --username root \> --password '******' \> --table clusters \> --target-dir=/tmp/sqoop_test \> --delete-target-dir \> --num-mappers 2 \> --split-by cluster_id \> --fields-terminated-by '\t'
Warning: /usr/lib/sqoop/bin/../../hbase does not exist! HBase imports will fail.Please set $HBASE_HOME to the root of your HBase installation.Warning: /usr/lib/sqoop/bin/../../hcatalog does not exist! HCatalog jobs will fail.Please set $HCAT_HOME to the root of your HCatalog installation.Warning: /usr/lib/sqoop/bin/../../accumulo does not exist! Accumulo imports will fail.Please set $ACCUMULO_HOME to the root of your Accumulo installation.2022-07-12 19:53:26,021 INFO sqoop.Sqoop: Running Sqoop version: 1.4.72022-07-12 19:53:26,104 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/lang/StringUtils    at org.apache.sqoop.tool.BaseSqoopTool.validateHiveOptions(BaseSqoopTool.java:1583)    at org.apache.sqoop.tool.ImportTool.validateOptions(ImportTool.java:1178)    at org.apache.sqoop.Sqoop.run(Sqoop.java:137)    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)    at org.apache.sqoop.Sqoop.main(Sqoop.java:252)Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang.StringUtils    at java.net.URLClassLoader.findClass(URLClassLoader.java:382)    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)    ... 8 more 

This problem was solved when I downloaded commons-lang-2.6.jar to /usr/lib/sqoop/lib/.
 
The following is the problem recurrence step:

//代码占位符
# 1. download bigtop.repo
wget -O /etc/yum.repos.d/bigtop.repo https://downloads.apache.org/bigtop/bigtop-3.1.0/repos/centos-7/bigtop.repo 
# 2. install sqoop
yum install sqoop -y
# 3. Execute this shell
/usr/lib/sqoop/bin/sqoop import \
--connect jdbc:mysql://bdp81:3306/ambari \
--username root \
--password '******' \
--table clusters \
--target-dir=/tmp/sqoop_test \
--delete-target-dir \
--num-mappers 2 \
--split-by cluster_id \
--fields-terminated-by '\t' 

Thanks in advance.
 

Issue Links

• is related to: SQOOP-3474 Sqoop import failed "java.lang.ClassNotFoundException: org.apache.commons.lang.StringUtils"
• links to: https://github.com/apache/bigtop/pull/944