[SQOOP-2896] Sqoop exec job fails with SQLException Access denied for user

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.4.7
• Assignee: Sowmya Ramesh
• Reporter: Kavan Suresh
• Created: 28/Mar/16 18:07
• Updated: 30/Mar/16 02:39
• Resolved: 30/Mar/16 01:03

Description

Sqoop save jobs which records the config information required to execute a Sqoop command have been failing on recent builds with an access denied exception. The default behavior is to create a job and then invoke the job with exec job command by passing username and password. In the recent builds of HDP 2.4 the parameter is not being passed when executing the exec command as seen in the below logs:

RUNNING: /usr/hdp/current/sqoop-client/bin/sqoop job --create myjob -- eval --driver com.mysql.jdbc.Driver --connect jdbc:mysql://hn0-sqoop.sqoop-hadl24-ssh.c7.internal.cloudapp.net/employees -e "select e.emp_no, e.first_name, e.last_name from employees as e limit 10"  --username sqoop --password sqoop
2016-03-22 22:15:02,464|beaver.machine|INFO|3182|139761292187392|Thread-4|Warning: /usr/hdp/2.4.2.0-29/accumulo does not exist! Accumulo imports will fail.
2016-03-22 22:15:02,464|beaver.machine|INFO|3182|139761292187392|Thread-4|Please set $ACCUMULO_HOME to the root of your Accumulo installation.
2016-03-22 22:15:05,815|beaver.machine|INFO|3182|139761283794688|Thread-5|16/03/22 22:15:05 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.4.2.0-29
2016-03-22 22:15:06,682|beaver.machine|INFO|3182|139761283794688|Thread-5|SLF4J: Class path contains multiple SLF4J bindings.
2016-03-22 22:15:06,683|beaver.machine|INFO|3182|139761283794688|Thread-5|SLF4J: Found binding in [jar:file:/usr/hdp/2.4.2.0-29/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
2016-03-22 22:15:06,683|beaver.machine|INFO|3182|139761283794688|Thread-5|SLF4J: Found binding in [jar:file:/usr/hdp/2.4.2.0-29/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
2016-03-22 22:15:06,683|beaver.machine|INFO|3182|139761283794688|Thread-5|SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
2016-03-22 22:15:06,693|beaver.machine|INFO|3182|139761283794688|Thread-5|SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2016-03-22 22:15:07,710|beaver.machine|INFO|3182|139761283794688|Thread-5|16/03/22 22:15:07 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
2016-03-22 22:15:08,550|beaver.machine|INFO|3182|139761510102848|MainThread|RUNNING: /usr/hdp/current/sqoop-client/bin/sqoop job --list
2016-03-22 22:15:08,572|beaver.machine|INFO|3182|139761292187392|Thread-6|Warning: /usr/hdp/2.4.2.0-29/accumulo does not exist! Accumulo imports will fail.
2016-03-22 22:15:08,573|beaver.machine|INFO|3182|139761292187392|Thread-6|Please set $ACCUMULO_HOME to the root of your Accumulo installation.
2016-03-22 22:15:12,014|beaver.machine|INFO|3182|139761283794688|Thread-7|16/03/22 22:15:12 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.4.2.0-29
2016-03-22 22:15:12,855|beaver.machine|INFO|3182|139761283794688|Thread-7|SLF4J: Class path contains multiple SLF4J bindings.
2016-03-22 22:15:12,855|beaver.machine|INFO|3182|139761283794688|Thread-7|SLF4J: Found binding in [jar:file:/usr/hdp/2.4.2.0-29/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
2016-03-22 22:15:12,856|beaver.machine|INFO|3182|139761283794688|Thread-7|SLF4J: Found binding in [jar:file:/usr/hdp/2.4.2.0-29/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
2016-03-22 22:15:12,856|beaver.machine|INFO|3182|139761283794688|Thread-7|SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
2016-03-22 22:15:12,864|beaver.machine|INFO|3182|139761283794688|Thread-7|SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2016-03-22 22:15:13,961|beaver.machine|INFO|3182|139761292187392|Thread-6|Available jobs:
2016-03-22 22:15:13,962|beaver.machine|INFO|3182|139761292187392|Thread-6|myjob

RUNNING: /usr/hdp/current/sqoop-client/bin/sqoop job --exec myjob -- --username sqoop --password sqoop
2016-03-22 22:15:22,783|beaver.machine|INFO|3182|139761283794688|Thread-10|Warning: /usr/hdp/2.4.2.0-29/accumulo does not exist! Accumulo imports will fail.
2016-03-22 22:15:22,783|beaver.machine|INFO|3182|139761283794688|Thread-10|Please set $ACCUMULO_HOME to the root of your Accumulo installation.
2016-03-22 22:15:27,442|beaver.machine|INFO|3182|139761292187392|Thread-11|16/03/22 22:15:27 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.4.2.0-29
2016-03-22 22:15:28,531|beaver.machine|INFO|3182|139761292187392|Thread-11|SLF4J: Class path contains multiple SLF4J bindings.
2016-03-22 22:15:28,531|beaver.machine|INFO|3182|139761292187392|Thread-11|SLF4J: Found binding in [jar:file:/usr/hdp/2.4.2.0-29/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
2016-03-22 22:15:28,531|beaver.machine|INFO|3182|139761292187392|Thread-11|SLF4J: Found binding in [jar:file:/usr/hdp/2.4.2.0-29/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
2016-03-22 22:15:28,531|beaver.machine|INFO|3182|139761292187392|Thread-11|SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
2016-03-22 22:15:28,542|beaver.machine|INFO|3182|139761292187392|Thread-11|SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2016-03-22 22:15:30,073|beaver.machine|INFO|3182|139761292187392|Thread-11|16/03/22 22:15:30 ERROR sqoop.SqoopOptions: It seems that you have launched a Sqoop metastore job via
2016-03-22 22:15:30,083|beaver.machine|INFO|3182|139761292187392|Thread-11|16/03/22 22:15:30 ERROR sqoop.SqoopOptions: Oozie with sqoop.metastore.client.record.password disabled.
2016-03-22 22:15:30,083|beaver.machine|INFO|3182|139761292187392|Thread-11|16/03/22 22:15:30 ERROR sqoop.SqoopOptions: But this configuration is not supported because Sqoop can't
2016-03-22 22:15:30,084|beaver.machine|INFO|3182|139761292187392|Thread-11|16/03/22 22:15:30 ERROR sqoop.SqoopOptions: prompt the user to enter the password while being executed
2016-03-22 22:15:30,093|beaver.machine|INFO|3182|139761292187392|Thread-11|16/03/22 22:15:30 ERROR sqoop.SqoopOptions: as Oozie tasks. Please enable sqoop.metastore.client.record
2016-03-22 22:15:30,093|beaver.machine|INFO|3182|139761292187392|Thread-11|16/03/22 22:15:30 ERROR sqoop.SqoopOptions: .password in sqoop-site.xml, or provide the password
2016-03-22 22:15:30,093|beaver.machine|INFO|3182|139761292187392|Thread-11|16/03/22 22:15:30 ERROR sqoop.SqoopOptions: explicitly using --password in the command tag of the Oozie
2016-03-22 22:15:30,094|beaver.machine|INFO|3182|139761292187392|Thread-11|16/03/22 22:15:30 ERROR sqoop.SqoopOptions: workflow file.
2016-03-22 22:15:30,229|beaver.machine|INFO|3182|139761292187392|Thread-11|16/03/22 22:15:30 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.
2016-03-22 22:15:30,264|beaver.machine|INFO|3182|139761292187392|Thread-11|16/03/22 22:15:30 INFO manager.SqlManager: Using default fetchSize of 1000
2016-03-22 22:15:30,769|beaver.machine|INFO|3182|139761292187392|Thread-11|16/03/22 22:15:30 WARN tool.EvalSqlTool: SQL exception executing statement: java.sql.SQLException: Access denied for user 'sqoop'@'10.0.0.26' (using password: NO)

Issue Links

• links to: RB link