Uploaded image for project: 'Sqoop (Retired)'
  1. Sqoop (Retired)
  2. SQOOP-524

hsqldb jar update regression

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • None
    • 1.4.2
    • None
    • None

    Description

      This is due to recent update of hsqldb jar version. Updating the jar does not break the build but it breaks the execution of every SQOOP job, as Hadoop's classpath is prepended to Sqoop's classpath and hadoop contains previous version of hsqldb JAR.
      This is a class loading issue. hsqldb-1.8.0.10.jar from hadoop is loaded first and we have hsqldb-2.2.8.jar specific code in sqoop.

      Steps to reproduce:
      1. Checkout latest revision from SVN/GIT
      2. Build - "ant -Dhadoopversion <as_per_hadoop_version_you_have>"
      3. Try to run sqoop with any DBMS. You get -

      Exception in thread "main" java.lang.NoSuchMethodError: org.hsqldb.DatabaseURL.parseURL(Ljava/lang/String;ZZ)Lorg/hsqldb/persist/HsqlProperties;
      at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)
      at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source)
      at java.sql.DriverManager.getConnection(DriverManager.java:582)
      at java.sql.DriverManager.getConnection(DriverManager.java:185)
      at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:665)
      at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
      at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:525)
      at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:548)
      at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:191)
      at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:175)
      at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:262)
      at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1235)
      at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1060)
      at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
      at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390)
      at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
      at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
      at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
      at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
      at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
      at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
      at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)

      Unless, you set this on command-line:
      export HADOOP_USER_CLASSPATH_FIRST=true

      Haven't tried ahead but it should also fail when Sqoop internally triggers a Hadoop JOB. This is just a speculation, will confirm this.

      Attachments

        1. SQOOP-524.patch
          0.5 kB
          Abhijeet Gaikwad
        2. SQOOP-524 -1.patch
          1 kB
          Abhijeet Gaikwad
        3. SQOOP-524-2.patch
          2 kB
          Abhijeet Gaikwad

        Issue Links

          Activity

            People

              abhijeet_gaikwad Abhijeet Gaikwad
              abhijeet_gaikwad Abhijeet Gaikwad
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: