Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
Sqoop save jobs which records the config information required to execute a Sqoop command have been failing on recent builds with an access denied exception. The default behavior is to create a job and then invoke the job with exec job command by passing username and password. In the recent builds of HDP 2.4 the parameter is not being passed when executing the exec command as seen in the below logs:
RUNNING: /usr/hdp/current/sqoop-client/bin/sqoop job --create myjob -- eval --driver com.mysql.jdbc.Driver --connect jdbc:mysql://hn0-sqoop.sqoop-hadl24-ssh.c7.internal.cloudapp.net/employees -e "select e.emp_no, e.first_name, e.last_name from employees as e limit 10" --username sqoop --password sqoop 2016-03-22 22:15:02,464|beaver.machine|INFO|3182|139761292187392|Thread-4|Warning: /usr/hdp/2.4.2.0-29/accumulo does not exist! Accumulo imports will fail. 2016-03-22 22:15:02,464|beaver.machine|INFO|3182|139761292187392|Thread-4|Please set $ACCUMULO_HOME to the root of your Accumulo installation. 2016-03-22 22:15:05,815|beaver.machine|INFO|3182|139761283794688|Thread-5|16/03/22 22:15:05 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.4.2.0-29 2016-03-22 22:15:06,682|beaver.machine|INFO|3182|139761283794688|Thread-5|SLF4J: Class path contains multiple SLF4J bindings. 2016-03-22 22:15:06,683|beaver.machine|INFO|3182|139761283794688|Thread-5|SLF4J: Found binding in [jar:file:/usr/hdp/2.4.2.0-29/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] 2016-03-22 22:15:06,683|beaver.machine|INFO|3182|139761283794688|Thread-5|SLF4J: Found binding in [jar:file:/usr/hdp/2.4.2.0-29/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] 2016-03-22 22:15:06,683|beaver.machine|INFO|3182|139761283794688|Thread-5|SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. 2016-03-22 22:15:06,693|beaver.machine|INFO|3182|139761283794688|Thread-5|SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 2016-03-22 22:15:07,710|beaver.machine|INFO|3182|139761283794688|Thread-5|16/03/22 22:15:07 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 2016-03-22 22:15:08,550|beaver.machine|INFO|3182|139761510102848|MainThread|RUNNING: /usr/hdp/current/sqoop-client/bin/sqoop job --list 2016-03-22 22:15:08,572|beaver.machine|INFO|3182|139761292187392|Thread-6|Warning: /usr/hdp/2.4.2.0-29/accumulo does not exist! Accumulo imports will fail. 2016-03-22 22:15:08,573|beaver.machine|INFO|3182|139761292187392|Thread-6|Please set $ACCUMULO_HOME to the root of your Accumulo installation. 2016-03-22 22:15:12,014|beaver.machine|INFO|3182|139761283794688|Thread-7|16/03/22 22:15:12 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.4.2.0-29 2016-03-22 22:15:12,855|beaver.machine|INFO|3182|139761283794688|Thread-7|SLF4J: Class path contains multiple SLF4J bindings. 2016-03-22 22:15:12,855|beaver.machine|INFO|3182|139761283794688|Thread-7|SLF4J: Found binding in [jar:file:/usr/hdp/2.4.2.0-29/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] 2016-03-22 22:15:12,856|beaver.machine|INFO|3182|139761283794688|Thread-7|SLF4J: Found binding in [jar:file:/usr/hdp/2.4.2.0-29/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] 2016-03-22 22:15:12,856|beaver.machine|INFO|3182|139761283794688|Thread-7|SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. 2016-03-22 22:15:12,864|beaver.machine|INFO|3182|139761283794688|Thread-7|SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 2016-03-22 22:15:13,961|beaver.machine|INFO|3182|139761292187392|Thread-6|Available jobs: 2016-03-22 22:15:13,962|beaver.machine|INFO|3182|139761292187392|Thread-6|myjob
RUNNING: /usr/hdp/current/sqoop-client/bin/sqoop job --exec myjob -- --username sqoop --password sqoop 2016-03-22 22:15:22,783|beaver.machine|INFO|3182|139761283794688|Thread-10|Warning: /usr/hdp/2.4.2.0-29/accumulo does not exist! Accumulo imports will fail. 2016-03-22 22:15:22,783|beaver.machine|INFO|3182|139761283794688|Thread-10|Please set $ACCUMULO_HOME to the root of your Accumulo installation. 2016-03-22 22:15:27,442|beaver.machine|INFO|3182|139761292187392|Thread-11|16/03/22 22:15:27 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.4.2.0-29 2016-03-22 22:15:28,531|beaver.machine|INFO|3182|139761292187392|Thread-11|SLF4J: Class path contains multiple SLF4J bindings. 2016-03-22 22:15:28,531|beaver.machine|INFO|3182|139761292187392|Thread-11|SLF4J: Found binding in [jar:file:/usr/hdp/2.4.2.0-29/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] 2016-03-22 22:15:28,531|beaver.machine|INFO|3182|139761292187392|Thread-11|SLF4J: Found binding in [jar:file:/usr/hdp/2.4.2.0-29/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] 2016-03-22 22:15:28,531|beaver.machine|INFO|3182|139761292187392|Thread-11|SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. 2016-03-22 22:15:28,542|beaver.machine|INFO|3182|139761292187392|Thread-11|SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 2016-03-22 22:15:30,073|beaver.machine|INFO|3182|139761292187392|Thread-11|16/03/22 22:15:30 ERROR sqoop.SqoopOptions: It seems that you have launched a Sqoop metastore job via 2016-03-22 22:15:30,083|beaver.machine|INFO|3182|139761292187392|Thread-11|16/03/22 22:15:30 ERROR sqoop.SqoopOptions: Oozie with sqoop.metastore.client.record.password disabled. 2016-03-22 22:15:30,083|beaver.machine|INFO|3182|139761292187392|Thread-11|16/03/22 22:15:30 ERROR sqoop.SqoopOptions: But this configuration is not supported because Sqoop can't 2016-03-22 22:15:30,084|beaver.machine|INFO|3182|139761292187392|Thread-11|16/03/22 22:15:30 ERROR sqoop.SqoopOptions: prompt the user to enter the password while being executed 2016-03-22 22:15:30,093|beaver.machine|INFO|3182|139761292187392|Thread-11|16/03/22 22:15:30 ERROR sqoop.SqoopOptions: as Oozie tasks. Please enable sqoop.metastore.client.record 2016-03-22 22:15:30,093|beaver.machine|INFO|3182|139761292187392|Thread-11|16/03/22 22:15:30 ERROR sqoop.SqoopOptions: .password in sqoop-site.xml, or provide the password 2016-03-22 22:15:30,093|beaver.machine|INFO|3182|139761292187392|Thread-11|16/03/22 22:15:30 ERROR sqoop.SqoopOptions: explicitly using --password in the command tag of the Oozie 2016-03-22 22:15:30,094|beaver.machine|INFO|3182|139761292187392|Thread-11|16/03/22 22:15:30 ERROR sqoop.SqoopOptions: workflow file. 2016-03-22 22:15:30,229|beaver.machine|INFO|3182|139761292187392|Thread-11|16/03/22 22:15:30 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time. 2016-03-22 22:15:30,264|beaver.machine|INFO|3182|139761292187392|Thread-11|16/03/22 22:15:30 INFO manager.SqlManager: Using default fetchSize of 1000 2016-03-22 22:15:30,769|beaver.machine|INFO|3182|139761292187392|Thread-11|16/03/22 22:15:30 WARN tool.EvalSqlTool: SQL exception executing statement: java.sql.SQLException: Access denied for user 'sqoop'@'10.0.0.26' (using password: NO)
Attachments
Attachments
Issue Links
- links to