Sqoop
  1. Sqoop
  2. SQOOP-329

SQOOP doesn't work with the DB2 JCC driver

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.4.0-incubating
    • Component/s: None
    • Labels:
      None

      Description

      SQOOP doesn't work with the DB2 JCC driver. I needed to write some code.

      Just need to override these two methods:
      getDriverClass --> "com.ibm.db2.jcc.DB2Driver"

      getCurTimestampQuery --> Return "SELECT CURRENT TIMESTAMP FROM sysibm.sysdummy1 WITH UR;"

      1. SQOOP-329.patch
        13 kB
        Arvind Prabhakar
      2. SQOOP-329-1.patch
        13 kB
        Arvind Prabhakar

        Activity

        Hide
        Bilung Lee added a comment -

        Patch committed. Thanks, Arvind!

        Show
        Bilung Lee added a comment - Patch committed. Thanks, Arvind!
        Hide
        Hudson added a comment -

        Integrated in Sqoop-jdk-1.6 #27 (See https://builds.apache.org/job/Sqoop-jdk-1.6/27/)
        SQOOP-329: SQOOP doesn't work with the DB2 JCC driver

        (Arvind Prabhakar via Bilung Lee)

        blee : http://svn.apache.org/viewvc/?view=rev&rev=1173471
        Files :

        • /incubator/sqoop/trunk/COMPILING.txt
        • /incubator/sqoop/trunk/src/java/com/cloudera/sqoop/manager/Db2Manager.java
        • /incubator/sqoop/trunk/src/java/com/cloudera/sqoop/manager/DefaultManagerFactory.java
        • /incubator/sqoop/trunk/src/test/com/cloudera/sqoop/manager/DB2ManagerImportManualTest.java
        Show
        Hudson added a comment - Integrated in Sqoop-jdk-1.6 #27 (See https://builds.apache.org/job/Sqoop-jdk-1.6/27/ ) SQOOP-329 : SQOOP doesn't work with the DB2 JCC driver (Arvind Prabhakar via Bilung Lee) blee : http://svn.apache.org/viewvc/?view=rev&rev=1173471 Files : /incubator/sqoop/trunk/COMPILING.txt /incubator/sqoop/trunk/src/java/com/cloudera/sqoop/manager/Db2Manager.java /incubator/sqoop/trunk/src/java/com/cloudera/sqoop/manager/DefaultManagerFactory.java /incubator/sqoop/trunk/src/test/com/cloudera/sqoop/manager/DB2ManagerImportManualTest.java
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/1983/#review1990
        -----------------------------------------------------------

        Ship it!

        +1

        • Bilung

        On 2011-09-21 02:08:31, Arvind Prabhakar wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/1983/

        -----------------------------------------------------------

        (Updated 2011-09-21 02:08:31)

        Review request for Sqoop and Bilung Lee.

        Summary

        -------

        Added Db2Manager implementation along with a simple import test that is currently categorized as manual test.

        This addresses bug SQOOP-329.

        https://issues.apache.org/jira/browse/SQOOP-329

        Diffs

        -----

        /COMPILING.txt 1172789

        /src/java/com/cloudera/sqoop/manager/Db2Manager.java PRE-CREATION

        /src/java/com/cloudera/sqoop/manager/DefaultManagerFactory.java 1172789

        /src/test/com/cloudera/sqoop/manager/DB2ManagerImportManualTest.java PRE-CREATION

        Diff: https://reviews.apache.org/r/1983/diff

        Testing

        -------

        Ran all the unit, thirdparty, and manual tests. Checkstyle/findbugs did not introduce new warnings.

        Thanks,

        Arvind

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1983/#review1990 ----------------------------------------------------------- Ship it! +1 Bilung On 2011-09-21 02:08:31, Arvind Prabhakar wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1983/ ----------------------------------------------------------- (Updated 2011-09-21 02:08:31) Review request for Sqoop and Bilung Lee. Summary ------- Added Db2Manager implementation along with a simple import test that is currently categorized as manual test. This addresses bug SQOOP-329 . https://issues.apache.org/jira/browse/SQOOP-329 Diffs ----- /COMPILING.txt 1172789 /src/java/com/cloudera/sqoop/manager/Db2Manager.java PRE-CREATION /src/java/com/cloudera/sqoop/manager/DefaultManagerFactory.java 1172789 /src/test/com/cloudera/sqoop/manager/DB2ManagerImportManualTest.java PRE-CREATION Diff: https://reviews.apache.org/r/1983/diff Testing ------- Ran all the unit, thirdparty, and manual tests. Checkstyle/findbugs did not introduce new warnings. Thanks, Arvind
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/1983/
        -----------------------------------------------------------

        (Updated 2011-09-21 02:08:31.850689)

        Review request for Sqoop and Bilung Lee.

        Changes
        -------

        Thanks for the feedback Bilung. I have updated the patch accordingly.

        Summary
        -------

        Added Db2Manager implementation along with a simple import test that is currently categorized as manual test.

        This addresses bug SQOOP-329.
        https://issues.apache.org/jira/browse/SQOOP-329

        Diffs (updated)


        /COMPILING.txt 1172789
        /src/java/com/cloudera/sqoop/manager/Db2Manager.java PRE-CREATION
        /src/java/com/cloudera/sqoop/manager/DefaultManagerFactory.java 1172789
        /src/test/com/cloudera/sqoop/manager/DB2ManagerImportManualTest.java PRE-CREATION

        Diff: https://reviews.apache.org/r/1983/diff

        Testing
        -------

        Ran all the unit, thirdparty, and manual tests. Checkstyle/findbugs did not introduce new warnings.

        Thanks,

        Arvind

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1983/ ----------------------------------------------------------- (Updated 2011-09-21 02:08:31.850689) Review request for Sqoop and Bilung Lee. Changes ------- Thanks for the feedback Bilung. I have updated the patch accordingly. Summary ------- Added Db2Manager implementation along with a simple import test that is currently categorized as manual test. This addresses bug SQOOP-329 . https://issues.apache.org/jira/browse/SQOOP-329 Diffs (updated) /COMPILING.txt 1172789 /src/java/com/cloudera/sqoop/manager/Db2Manager.java PRE-CREATION /src/java/com/cloudera/sqoop/manager/DefaultManagerFactory.java 1172789 /src/test/com/cloudera/sqoop/manager/DB2ManagerImportManualTest.java PRE-CREATION Diff: https://reviews.apache.org/r/1983/diff Testing ------- Ran all the unit, thirdparty, and manual tests. Checkstyle/findbugs did not introduce new warnings. Thanks, Arvind
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/1983/#review1989
        -----------------------------------------------------------

        Thanks for the patch! One thing though. Those catalog queries (except for the current timestamp one) are specific to DB2 LUW and thus they may not work for other platforms (like zOS) of DB2.

        /src/java/com/cloudera/sqoop/manager/Db2Manager.java
        <https://reviews.apache.org/r/1983/#comment4481>

        So you may want to consider not to inherit from CatalogQueryManager. For listDatabases(), you could use DatabaseMetaData.getSchemas() for cross-platform support. For listTables(), getColumnNames(), and getPrimaryKey(), parent methods in SqlManager would be supported across platforms since they use generic JDBC metadata APIs.

        • Bilung

        On 2011-09-20 17:30:46, Arvind Prabhakar wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/1983/

        -----------------------------------------------------------

        (Updated 2011-09-20 17:30:46)

        Review request for Sqoop and Bilung Lee.

        Summary

        -------

        Added Db2Manager implementation along with a simple import test that is currently categorized as manual test.

        This addresses bug SQOOP-329.

        https://issues.apache.org/jira/browse/SQOOP-329

        Diffs

        -----

        /COMPILING.txt 1172789

        /src/java/com/cloudera/sqoop/manager/Db2Manager.java PRE-CREATION

        /src/java/com/cloudera/sqoop/manager/DefaultManagerFactory.java 1172789

        /src/test/com/cloudera/sqoop/manager/DB2ManagerImportManualTest.java PRE-CREATION

        Diff: https://reviews.apache.org/r/1983/diff

        Testing

        -------

        Ran all the unit, thirdparty, and manual tests. Checkstyle/findbugs did not introduce new warnings.

        Thanks,

        Arvind

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1983/#review1989 ----------------------------------------------------------- Thanks for the patch! One thing though. Those catalog queries (except for the current timestamp one) are specific to DB2 LUW and thus they may not work for other platforms (like zOS) of DB2. /src/java/com/cloudera/sqoop/manager/Db2Manager.java < https://reviews.apache.org/r/1983/#comment4481 > So you may want to consider not to inherit from CatalogQueryManager. For listDatabases(), you could use DatabaseMetaData.getSchemas() for cross-platform support. For listTables(), getColumnNames(), and getPrimaryKey(), parent methods in SqlManager would be supported across platforms since they use generic JDBC metadata APIs. Bilung On 2011-09-20 17:30:46, Arvind Prabhakar wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1983/ ----------------------------------------------------------- (Updated 2011-09-20 17:30:46) Review request for Sqoop and Bilung Lee. Summary ------- Added Db2Manager implementation along with a simple import test that is currently categorized as manual test. This addresses bug SQOOP-329 . https://issues.apache.org/jira/browse/SQOOP-329 Diffs ----- /COMPILING.txt 1172789 /src/java/com/cloudera/sqoop/manager/Db2Manager.java PRE-CREATION /src/java/com/cloudera/sqoop/manager/DefaultManagerFactory.java 1172789 /src/test/com/cloudera/sqoop/manager/DB2ManagerImportManualTest.java PRE-CREATION Diff: https://reviews.apache.org/r/1983/diff Testing ------- Ran all the unit, thirdparty, and manual tests. Checkstyle/findbugs did not introduce new warnings. Thanks, Arvind
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/1983/
        -----------------------------------------------------------

        Review request for Sqoop and Bilung Lee.

        Summary
        -------

        Added Db2Manager implementation along with a simple import test that is currently categorized as manual test.

        This addresses bug SQOOP-329.
        https://issues.apache.org/jira/browse/SQOOP-329

        Diffs


        /COMPILING.txt 1172789
        /src/java/com/cloudera/sqoop/manager/Db2Manager.java PRE-CREATION
        /src/java/com/cloudera/sqoop/manager/DefaultManagerFactory.java 1172789
        /src/test/com/cloudera/sqoop/manager/DB2ManagerImportManualTest.java PRE-CREATION

        Diff: https://reviews.apache.org/r/1983/diff

        Testing
        -------

        Ran all the unit, thirdparty, and manual tests. Checkstyle/findbugs did not introduce new warnings.

        Thanks,

        Arvind

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1983/ ----------------------------------------------------------- Review request for Sqoop and Bilung Lee. Summary ------- Added Db2Manager implementation along with a simple import test that is currently categorized as manual test. This addresses bug SQOOP-329 . https://issues.apache.org/jira/browse/SQOOP-329 Diffs /COMPILING.txt 1172789 /src/java/com/cloudera/sqoop/manager/Db2Manager.java PRE-CREATION /src/java/com/cloudera/sqoop/manager/DefaultManagerFactory.java 1172789 /src/test/com/cloudera/sqoop/manager/DB2ManagerImportManualTest.java PRE-CREATION Diff: https://reviews.apache.org/r/1983/diff Testing ------- Ran all the unit, thirdparty, and manual tests. Checkstyle/findbugs did not introduce new warnings. Thanks, Arvind
        Hide
        Arvind Prabhakar added a comment -

        First cut of Db2Manager implementation.

        Show
        Arvind Prabhakar added a comment - First cut of Db2Manager implementation.

          People

          • Assignee:
            Arvind Prabhakar
            Reporter:
            Brock Noland
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development