Sqoop
  1. Sqoop
  2. SQOOP-906

Sqoop is always calling ConnectionManager.datetimeToQueryString with TIMESTAMP column type

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.2
    • Fix Version/s: 1.4.4
    • Component/s: None
    • Labels:

      Description

      One user on CDH mailing list has risen issue with incremental import using lastmodified mode. Sqoop seems to be generating queries including TO_TIMESTAMP function even for column that is defined as Types.DATE. This is causing issues with Oracle Connector. The connector seems to be correctly written as appropriate OracleManager.datetimeToQueryString seems to be expected condition based on the column type. Unfortunately the caller code is always passing in Types.TIMESTAMP value instead of the original value. I think that we should fix that.

      1. SQOOP-906.4.txt
        14 kB
        Raghav Kumar Gautam
      2. SQOOP-906.txt
        11 kB
        Raghav Kumar Gautam
      3. SQOOP-906.txt
        13 kB
        Raghav Kumar Gautam
      4. SQOOP-906.txt
        5 kB
        Raghav Kumar Gautam

        Issue Links

          Activity

          Hide
          Raghav Kumar Gautam added a comment -

          Updated patch.

          Show
          Raghav Kumar Gautam added a comment - Updated patch.
          Hide
          ASF subversion and git services added a comment -

          Commit 5495113781286de00473eb3d8c535f4288454082 in branch refs/heads/trunk from Jarek Jarcec Cecho
          [ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=5495113 ]

          SQOOP-906: Sqoop is always calling ConnectionManager.datetimeToQueryString with TIMESTAMP column type

          (Raghav Kumar Gautam via Jarek Jarcec Cecho)

          Show
          ASF subversion and git services added a comment - Commit 5495113781286de00473eb3d8c535f4288454082 in branch refs/heads/trunk from Jarek Jarcec Cecho [ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=5495113 ] SQOOP-906 : Sqoop is always calling ConnectionManager.datetimeToQueryString with TIMESTAMP column type (Raghav Kumar Gautam via Jarek Jarcec Cecho)
          Hide
          Jarek Jarcec Cecho added a comment -

          Thank you Raghav Kumar Gautam for fixing this one!

          Show
          Jarek Jarcec Cecho added a comment - Thank you Raghav Kumar Gautam for fixing this one!
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop200 #817 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/817/)
          SQOOP-906: Sqoop is always calling ConnectionManager.datetimeToQueryString with TIMESTAMP column type (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=5495113781286de00473eb3d8c535f4288454082)

          • src/test/org/apache/sqoop/manager/oracle/OracleIncrementalImportTest.java
          • src/test/com/cloudera/sqoop/ThirdPartyTests.java
          • src/java/org/apache/sqoop/manager/ConnManager.java
          • src/java/org/apache/sqoop/tool/ImportTool.java
          • src/test/com/cloudera/sqoop/TestIncrementalImport.java
          • src/test/com/cloudera/sqoop/testutil/BaseSqoopTestCase.java
          • src/test/com/cloudera/sqoop/TestMerge.java
          • src/java/org/apache/sqoop/manager/OracleManager.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop200 #817 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/817/ ) SQOOP-906 : Sqoop is always calling ConnectionManager.datetimeToQueryString with TIMESTAMP column type (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=5495113781286de00473eb3d8c535f4288454082 ) src/test/org/apache/sqoop/manager/oracle/OracleIncrementalImportTest.java src/test/com/cloudera/sqoop/ThirdPartyTests.java src/java/org/apache/sqoop/manager/ConnManager.java src/java/org/apache/sqoop/tool/ImportTool.java src/test/com/cloudera/sqoop/TestIncrementalImport.java src/test/com/cloudera/sqoop/testutil/BaseSqoopTestCase.java src/test/com/cloudera/sqoop/TestMerge.java src/java/org/apache/sqoop/manager/OracleManager.java
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop100 #778 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/778/)
          SQOOP-906: Sqoop is always calling ConnectionManager.datetimeToQueryString with TIMESTAMP column type (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=5495113781286de00473eb3d8c535f4288454082)

          • src/java/org/apache/sqoop/manager/OracleManager.java
          • src/java/org/apache/sqoop/tool/ImportTool.java
          • src/test/com/cloudera/sqoop/TestMerge.java
          • src/test/org/apache/sqoop/manager/oracle/OracleIncrementalImportTest.java
          • src/test/com/cloudera/sqoop/testutil/BaseSqoopTestCase.java
          • src/test/com/cloudera/sqoop/ThirdPartyTests.java
          • src/test/com/cloudera/sqoop/TestIncrementalImport.java
          • src/java/org/apache/sqoop/manager/ConnManager.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop100 #778 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/778/ ) SQOOP-906 : Sqoop is always calling ConnectionManager.datetimeToQueryString with TIMESTAMP column type (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=5495113781286de00473eb3d8c535f4288454082 ) src/java/org/apache/sqoop/manager/OracleManager.java src/java/org/apache/sqoop/tool/ImportTool.java src/test/com/cloudera/sqoop/TestMerge.java src/test/org/apache/sqoop/manager/oracle/OracleIncrementalImportTest.java src/test/com/cloudera/sqoop/testutil/BaseSqoopTestCase.java src/test/com/cloudera/sqoop/ThirdPartyTests.java src/test/com/cloudera/sqoop/TestIncrementalImport.java src/java/org/apache/sqoop/manager/ConnManager.java
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop23 #1019 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/1019/)
          SQOOP-906: Sqoop is always calling ConnectionManager.datetimeToQueryString with TIMESTAMP column type (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=5495113781286de00473eb3d8c535f4288454082)

          • src/test/com/cloudera/sqoop/testutil/BaseSqoopTestCase.java
          • src/java/org/apache/sqoop/manager/ConnManager.java
          • src/test/com/cloudera/sqoop/ThirdPartyTests.java
          • src/test/com/cloudera/sqoop/TestIncrementalImport.java
          • src/java/org/apache/sqoop/manager/OracleManager.java
          • src/java/org/apache/sqoop/tool/ImportTool.java
          • src/test/org/apache/sqoop/manager/oracle/OracleIncrementalImportTest.java
          • src/test/com/cloudera/sqoop/TestMerge.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop23 #1019 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/1019/ ) SQOOP-906 : Sqoop is always calling ConnectionManager.datetimeToQueryString with TIMESTAMP column type (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=5495113781286de00473eb3d8c535f4288454082 ) src/test/com/cloudera/sqoop/testutil/BaseSqoopTestCase.java src/java/org/apache/sqoop/manager/ConnManager.java src/test/com/cloudera/sqoop/ThirdPartyTests.java src/test/com/cloudera/sqoop/TestIncrementalImport.java src/java/org/apache/sqoop/manager/OracleManager.java src/java/org/apache/sqoop/tool/ImportTool.java src/test/org/apache/sqoop/manager/oracle/OracleIncrementalImportTest.java src/test/com/cloudera/sqoop/TestMerge.java
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop20 #819 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/819/)
          SQOOP-906: Sqoop is always calling ConnectionManager.datetimeToQueryString with TIMESTAMP column type (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=5495113781286de00473eb3d8c535f4288454082)

          • src/test/com/cloudera/sqoop/ThirdPartyTests.java
          • src/test/com/cloudera/sqoop/testutil/BaseSqoopTestCase.java
          • src/test/com/cloudera/sqoop/TestMerge.java
          • src/java/org/apache/sqoop/manager/ConnManager.java
          • src/java/org/apache/sqoop/tool/ImportTool.java
          • src/test/com/cloudera/sqoop/TestIncrementalImport.java
          • src/java/org/apache/sqoop/manager/OracleManager.java
          • src/test/org/apache/sqoop/manager/oracle/OracleIncrementalImportTest.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop20 #819 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/819/ ) SQOOP-906 : Sqoop is always calling ConnectionManager.datetimeToQueryString with TIMESTAMP column type (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=5495113781286de00473eb3d8c535f4288454082 ) src/test/com/cloudera/sqoop/ThirdPartyTests.java src/test/com/cloudera/sqoop/testutil/BaseSqoopTestCase.java src/test/com/cloudera/sqoop/TestMerge.java src/java/org/apache/sqoop/manager/ConnManager.java src/java/org/apache/sqoop/tool/ImportTool.java src/test/com/cloudera/sqoop/TestIncrementalImport.java src/java/org/apache/sqoop/manager/OracleManager.java src/test/org/apache/sqoop/manager/oracle/OracleIncrementalImportTest.java

            People

            • Assignee:
              Raghav Kumar Gautam
              Reporter:
              Jarek Jarcec Cecho
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development