Uploaded image for project: 'Sqoop (Retired)'
  1. Sqoop (Retired)
  2. SQOOP-2940

Sqoop fails to record password sometimes

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 1.4.6
    • None
    • sqoop2-api
    • None
    • Important

    Description

      I have my sqoop-site.xml configured to record passwords i. e

      <property>
      <name>sqoop.metastore.client.record.password</name>
      <value>true</value>
      <description>If true, allow saved passwords in the metastore.
      </description>
      </property>
      <property>

      Also, I created all my jobs after configuring it to record passwords.

      Now, it works alright initially but after running the job a couple of times from oozie, it starts asking for password for only a few of the jobs.

      For example, If we have 12 jobs as part of an oozie workflow, oozie job fails after executing 5-6 jobs.

      Though all these jobs were created after setting record password property to true, it throws an error:

      Error executing statement: java.sql.SQLException: Access denied for user 'root'@'IP' (using password: NO)
      java.sql.SQLException: Access denied for user 'root'@'IP' (using password: NO)
      at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1073)
      at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3597)
      at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3529)
      at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:935)
      at com.mysql.jdbc.MysqlIO.secureAuth411(MysqlIO.java:4101)
      at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1300)
      at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2337)
      at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2370)
      at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2154)
      at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:792)
      at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:49)
      at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
      at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
      at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
      at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:381)
      at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:305)
      at java.sql.DriverManager.getConnection(DriverManager.java:571)
      at java.sql.DriverManager.getConnection(DriverManager.java:215)
      at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:904)
      at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
      at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:763)
      at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:786)
      at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:289)
      at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:260)
      at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:246)
      at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:327)
      at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1846)
      at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1646)
      at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
      at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
      at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
      at org.apache.sqoop.tool.JobTool.execJob(JobTool.java:228)
      at org.apache.sqoop.tool.JobTool.run(JobTool.java:283)
      at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
      at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
      at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
      at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
      at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
      at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:197)
      at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:177)
      at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:49)
      at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:46)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:606)
      at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:236)
      at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
      at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
      at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:388)
      at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:302)
      at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:187)
      at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:230)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:745)
      6483 [uber-SubtaskRunner] ERROR org.apache.sqoop.tool.ImportTool - Encountered IOException running import job: java.io.IOException: No columns to generate for ClassWriter
      at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1652)
      at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
      at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
      at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
      at org.apache.sqoop.tool.JobTool.execJob(JobTool.java:228)
      at org.apache.sqoop.tool.JobTool.run(JobTool.java:283)
      at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
      at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
      at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
      at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
      at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
      at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:197)
      at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:177)
      at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:49)
      at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:46)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:606)
      at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:236)
      at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
      at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
      at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:388)
      at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:302)
      at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:187)
      at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:230)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:745)

      (using password NO) is the problem here. It is prompting for password and since the job is running through oozie (which isn't an interactive mode) and so password can not be supplied manually.

      Also, the command

      sqoop job --meta-connect jdbc:hsqldb:hsql://FQDN:16000/sqoop --show dispatch_job |grep db.require.password

      returns false and yet on trying to execute the job, it prompts for password on the console.

      Attachments

        Activity

          People

            Unassigned Unassigned
            yourtechchick simran
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: