Sqoop
  1. Sqoop
  2. SQOOP-476

If table name is a qualified name, Sqoop import fails in DB2

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.4.2
    • Component/s: None
    • Labels:
      None

      Description

      In DB2, it is OK to have a dot "." in table name, but it is not in alias. For example, the following query fails:

      SELECT * FROM foo.bar AS foo.bar
      
      SQL0104N  An unexpected token "." was found following "from foo.bar as foo".  
      Expected tokens may include:  ",".  SQLSTATE=42601
      

      During the import job, Sqoop uses the following select query:

      SELECT <column names> FROM <table name> AS <table name>
      

      As can be seen, it can be a problem for DB2 if table name has a dot "." in it.

      1. SQOOP-476.patch
        15 kB
        Cheolsoo Park

        Activity

        Hide
        Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6 #118 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6/118/)
        SQOOP-476 If table name is a qualified name, Sqoop import fails in DB2 (Revision 1345282)

        Result = SUCCESS
        blee :
        Files :

        • /sqoop/trunk/src/java/org/apache/sqoop/manager/Db2Manager.java
        • /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBInputFormat.java
        • /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java
        • /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/Db2DBRecordReader.java
        • /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBInputFormat.java
        • /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBRecordReader.java
        • /sqoop/trunk/src/test/com/cloudera/sqoop/manager/DB2ManagerImportManualTest.java
        Show
        Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6 #118 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6/118/ ) SQOOP-476 If table name is a qualified name, Sqoop import fails in DB2 (Revision 1345282) Result = SUCCESS blee : Files : /sqoop/trunk/src/java/org/apache/sqoop/manager/Db2Manager.java /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBInputFormat.java /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/Db2DBRecordReader.java /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBInputFormat.java /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBRecordReader.java /sqoop/trunk/src/test/com/cloudera/sqoop/manager/DB2ManagerImportManualTest.java
        Hide
        Bilung Lee added a comment -

        Patch is in. Thanks, Cheolsoo!

        Show
        Bilung Lee added a comment - Patch is in. Thanks, Cheolsoo!
        Hide
        Cheolsoo Park added a comment -
        Show
        Cheolsoo Park added a comment - Review board: https://reviews.apache.org/r/4674/
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4674/#review7917
        -----------------------------------------------------------

        Thanks for the patch! It looks good overall. One comment though. It seems that two DB2 record readers are
        provided, but only one of them is actually used by the Db2Manager and is tested by the test case. Unless the
        generic JDBC manager also works, there is probably no need for the other record reader.

        • Bilung

        On 2012-04-07 00:30:40, Cheolsoo Park wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4674/

        -----------------------------------------------------------

        (Updated 2012-04-07 00:30:40)

        Review request for Sqoop.

        Summary

        -------

        During the import job, Sqoop uses the following select query:

        SELECT <column names> FROM <table name> AS <table name>

        This causes a SQL syntax error for DB2 if table name has a dot "." in it.

        The changes include:

        1) Create DB2-specific DBInputFormat/DBRecordReader + DataDrivenDBInputFormat. Note that DataDrivenDBRecordReader is not subclassed since DB2-specific logic is implemented directly in its getSelectQuery() method.

        2) Add a unit test that imports data from a table that has a qualified name.

        This addresses bug SQOOP-476.

        https://issues.apache.org/jira/browse/SQOOP-476

        Diffs

        -----

        /src/java/org/apache/sqoop/mapreduce/db/DBInputFormat.java 1310644

        /src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java 1310644

        /src/java/org/apache/sqoop/mapreduce/db/Db2DBRecordReader.java PRE-CREATION

        /src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBInputFormat.java PRE-CREATION

        /src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBRecordReader.java PRE-CREATION

        /src/test/com/cloudera/sqoop/manager/DB2ManagerImportManualTest.java 1310644

        /src/java/org/apache/sqoop/manager/Db2Manager.java 1310644

        Diff: https://reviews.apache.org/r/4674/diff

        Testing

        -------

        Thanks,

        Cheolsoo

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4674/#review7917 ----------------------------------------------------------- Thanks for the patch! It looks good overall. One comment though. It seems that two DB2 record readers are provided, but only one of them is actually used by the Db2Manager and is tested by the test case. Unless the generic JDBC manager also works, there is probably no need for the other record reader. Bilung On 2012-04-07 00:30:40, Cheolsoo Park wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4674/ ----------------------------------------------------------- (Updated 2012-04-07 00:30:40) Review request for Sqoop. Summary ------- During the import job, Sqoop uses the following select query: SELECT <column names> FROM <table name> AS <table name> This causes a SQL syntax error for DB2 if table name has a dot "." in it. The changes include: 1) Create DB2-specific DBInputFormat/DBRecordReader + DataDrivenDBInputFormat. Note that DataDrivenDBRecordReader is not subclassed since DB2-specific logic is implemented directly in its getSelectQuery() method. 2) Add a unit test that imports data from a table that has a qualified name. This addresses bug SQOOP-476 . https://issues.apache.org/jira/browse/SQOOP-476 Diffs ----- /src/java/org/apache/sqoop/mapreduce/db/DBInputFormat.java 1310644 /src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java 1310644 /src/java/org/apache/sqoop/mapreduce/db/Db2DBRecordReader.java PRE-CREATION /src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBInputFormat.java PRE-CREATION /src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBRecordReader.java PRE-CREATION /src/test/com/cloudera/sqoop/manager/DB2ManagerImportManualTest.java 1310644 /src/java/org/apache/sqoop/manager/Db2Manager.java 1310644 Diff: https://reviews.apache.org/r/4674/diff Testing ------- Thanks, Cheolsoo
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4674/
        -----------------------------------------------------------

        Review request for Sqoop.

        Summary
        -------

        During the import job, Sqoop uses the following select query:

        SELECT <column names> FROM <table name> AS <table name>

        This causes a SQL syntax error for DB2 if table name has a dot "." in it.

        The changes include:

        1) Create DB2-specific DBInputFormat/DBRecordReader + DataDrivenDBInputFormat. Note that DataDrivenDBRecordReader is not subclassed since DB2-specific logic is implemented directly in its getSelectQuery() method.

        2) Add a unit test that imports data from a table that has a qualified name.

        This addresses bug SQOOP-476.
        https://issues.apache.org/jira/browse/SQOOP-476

        Diffs


        /src/java/org/apache/sqoop/mapreduce/db/DBInputFormat.java 1310644
        /src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java 1310644
        /src/java/org/apache/sqoop/mapreduce/db/Db2DBRecordReader.java PRE-CREATION
        /src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBInputFormat.java PRE-CREATION
        /src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBRecordReader.java PRE-CREATION
        /src/test/com/cloudera/sqoop/manager/DB2ManagerImportManualTest.java 1310644
        /src/java/org/apache/sqoop/manager/Db2Manager.java 1310644

        Diff: https://reviews.apache.org/r/4674/diff

        Testing
        -------

        Thanks,

        Cheolsoo

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4674/ ----------------------------------------------------------- Review request for Sqoop. Summary ------- During the import job, Sqoop uses the following select query: SELECT <column names> FROM <table name> AS <table name> This causes a SQL syntax error for DB2 if table name has a dot "." in it. The changes include: 1) Create DB2-specific DBInputFormat/DBRecordReader + DataDrivenDBInputFormat. Note that DataDrivenDBRecordReader is not subclassed since DB2-specific logic is implemented directly in its getSelectQuery() method. 2) Add a unit test that imports data from a table that has a qualified name. This addresses bug SQOOP-476 . https://issues.apache.org/jira/browse/SQOOP-476 Diffs /src/java/org/apache/sqoop/mapreduce/db/DBInputFormat.java 1310644 /src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java 1310644 /src/java/org/apache/sqoop/mapreduce/db/Db2DBRecordReader.java PRE-CREATION /src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBInputFormat.java PRE-CREATION /src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBRecordReader.java PRE-CREATION /src/test/com/cloudera/sqoop/manager/DB2ManagerImportManualTest.java 1310644 /src/java/org/apache/sqoop/manager/Db2Manager.java 1310644 Diff: https://reviews.apache.org/r/4674/diff Testing ------- Thanks, Cheolsoo

          People

          • Assignee:
            Cheolsoo Park
            Reporter:
            Cheolsoo Park
          • Votes:
            2 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development