Sqoop
  1. Sqoop
  2. SQOOP-1164

sqoop picks up hsqldb driver jar when connect to mysql db, and failed

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: 1.4.3
    • Fix Version/s: None
    • Component/s: tools
    • Labels:
      None
    • Environment:

      RHEL 5.4 x86_64

      Description

      Under SQOOP/lib, I have both hsqldb driver jar file and mysql driver jar file:

      • lib/hsqldb-1.8.0.10.jar
      • lib/mysql-connector-java-5.1.20-bin.jar

      But, when I try to connect to mysql db using sqoop import tool, sqoop will pick up hsqldb driver jar and returned following exception. Even if I add '--connection-manager org.apache.sqoop.manager.MySQLManager' into the command, this issue still exists. But, after I removed the hsqldb jar file from lib folder, this issue does not exist any more.

      > bin/sqoop import --connect jdbc:mysql://hostname:3306/test --connection-manager org.apache.sqoop.manager.MySQLManager --username user --password pwd --table testtbl -m 1 --target-dir /tmp/test5
      13/08/05 18:44:03 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
      13/08/05 18:44:03 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
      13/08/05 18:44:03 INFO tool.CodeGenTool: Beginning code generation
      Exception in thread "main" java.lang.NoSuchMethodError: org/hsqldb/DatabaseURL.parseURL(Ljava/lang/String;ZZ)Lorg/hsqldb/persist/HsqlProperties;
              at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)
              at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source)
              at java.sql.DriverManager.getConnection(DriverManager.java:402)
              at java.sql.DriverManager.getConnection(DriverManager.java:450)
              at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:745)
              at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
              at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:605)
              at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:628)
              at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:235)
              at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:219)
              at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:283)
              at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1255)
              at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1072)
              at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
              at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390)
              at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
              at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
              at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
              at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
              at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
              at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
              at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
      

      A possible solution is to add a new parameter in Sqoop import/export/... tools to specify the path of driver jar(s). The new parameter likes '--driverJar lib/mysql-connector-java-5.1.20-bin.jar'.

        Issue Links

          Activity

          Hide
          Andrew Otto added a comment -

          I had this same problem! Indeed, there was another (newer) version of hsqldb installed on my node. I'm running Ubuntu, and for some reason the package libhsqldb-java was installed, which was providing version 2.2.9. This was being automatically included in sqoop's CLASSPATH. I removed this package and all is well.

          Show
          Andrew Otto added a comment - I had this same problem! Indeed, there was another (newer) version of hsqldb installed on my node. I'm running Ubuntu, and for some reason the package libhsqldb-java was installed, which was providing version 2.2.9. This was being automatically included in sqoop's CLASSPATH. I removed this package and all is well.
          Hide
          Jarek Jarcec Cecho added a comment -

          Additional details are available on the user group mailing list.

          Show
          Jarek Jarcec Cecho added a comment - Additional details are available on the user group mailing list.
          Jarek Jarcec Cecho made changes -
          Remote Link This issue links to "Email discussion on user group (Web Link)" [ 12513 ]
          Jarek Jarcec Cecho made changes -
          Description Under SQOOP/lib, I have both hsqldb driver jar file and mysql driver jar file:
          - lib/hsqldb-1.8.0.10.jar
          - lib/mysql-connector-java-5.1.20-bin.jar

          But, when I try to connect to mysql db using sqoop import tool, sqoop will pick up hsqldb driver jar and returned following exception. Even if I add '--connection-manager org.apache.sqoop.manager.MySQLManager' into the command, this issue still exists. But, after I removed the hsqldb jar file from lib folder, this issue does not exist any more.

          > bin/sqoop import --connect jdbc:mysql://hostname:3306/test --connection-manager org.apache.sqoop.manager.MySQLManager --username user --password pwd --table testtbl -m 1 --target-dir /tmp/test5
          13/08/05 18:44:03 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
          13/08/05 18:44:03 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
          13/08/05 18:44:03 INFO tool.CodeGenTool: Beginning code generation
          Exception in thread "main" java.lang.NoSuchMethodError: org/hsqldb/DatabaseURL.parseURL(Ljava/lang/String;ZZ)Lorg/hsqldb/persist/HsqlProperties;
                  at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)
                  at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source)
                  at java.sql.DriverManager.getConnection(DriverManager.java:402)
                  at java.sql.DriverManager.getConnection(DriverManager.java:450)
                  at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:745)
                  at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
                  at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:605)
                  at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:628)
                  at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:235)
                  at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:219)
                  at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:283)
                  at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1255)
                  at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1072)
                  at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
                  at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390)
                  at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
                  at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
                  at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
                  at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
                  at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
                  at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
                  at org.apache.sqoop.Sqoop.main(Sqoop.java:238)

          A possible solution is to add a new parameter in Sqoop import/export/... tools to specify the path of driver jar(s). The new parameter likes '--driverJar lib/mysql-connector-java-5.1.20-bin.jar'.

          Under SQOOP/lib, I have both hsqldb driver jar file and mysql driver jar file:
          - lib/hsqldb-1.8.0.10.jar
          - lib/mysql-connector-java-5.1.20-bin.jar

          But, when I try to connect to mysql db using sqoop import tool, sqoop will pick up hsqldb driver jar and returned following exception. Even if I add '--connection-manager org.apache.sqoop.manager.MySQLManager' into the command, this issue still exists. But, after I removed the hsqldb jar file from lib folder, this issue does not exist any more.

          {code}
          > bin/sqoop import --connect jdbc:mysql://hostname:3306/test --connection-manager org.apache.sqoop.manager.MySQLManager --username user --password pwd --table testtbl -m 1 --target-dir /tmp/test5
          13/08/05 18:44:03 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
          13/08/05 18:44:03 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
          13/08/05 18:44:03 INFO tool.CodeGenTool: Beginning code generation
          Exception in thread "main" java.lang.NoSuchMethodError: org/hsqldb/DatabaseURL.parseURL(Ljava/lang/String;ZZ)Lorg/hsqldb/persist/HsqlProperties;
                  at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)
                  at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source)
                  at java.sql.DriverManager.getConnection(DriverManager.java:402)
                  at java.sql.DriverManager.getConnection(DriverManager.java:450)
                  at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:745)
                  at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
                  at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:605)
                  at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:628)
                  at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:235)
                  at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:219)
                  at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:283)
                  at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1255)
                  at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1072)
                  at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
                  at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390)
                  at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
                  at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
                  at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
                  at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
                  at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
                  at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
                  at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
          {code}

          A possible solution is to add a new parameter in Sqoop import/export/... tools to specify the path of driver jar(s). The new parameter likes '--driverJar lib/mysql-connector-java-5.1.20-bin.jar'.

          Jarek Jarcec Cecho made changes -
          Field Original Value New Value
          Link This issue is duplicated by SQOOP-1163 [ SQOOP-1163 ]
          Hide
          sam liu added a comment -

          I tried with --driver param, but it does not work for me, and returned the same exception.

          My command is 'bin/sqoop import --driver com.mysql.jdbc.Driver --connect jdbc:mysql://hostname:3306/test --username user --password pwd --table testtbl -m 1 --target-dir /tmp/test6'.

          Show
          sam liu added a comment - I tried with --driver param, but it does not work for me, and returned the same exception. My command is 'bin/sqoop import --driver com.mysql.jdbc.Driver --connect jdbc:mysql://hostname:3306/test --username user --password pwd --table testtbl -m 1 --target-dir /tmp/test6'.
          Hide
          Abraham Elmahrek added a comment -

          --driver parameter should resolve this?

          Show
          Abraham Elmahrek added a comment - --driver parameter should resolve this?
          sam liu created issue -

            People

            • Assignee:
              Unassigned
              Reporter:
              sam liu
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:

                Development