Sqoop
  1. Sqoop
  2. SQOOP-906

Sqoop is always calling ConnectionManager.datetimeToQueryString with TIMESTAMP column type

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.2
    • Fix Version/s: 1.4.4
    • Component/s: None
    • Labels:

      Description

      One user on CDH mailing list has risen issue with incremental import using lastmodified mode. Sqoop seems to be generating queries including TO_TIMESTAMP function even for column that is defined as Types.DATE. This is causing issues with Oracle Connector. The connector seems to be correctly written as appropriate OracleManager.datetimeToQueryString seems to be expected condition based on the column type. Unfortunately the caller code is always passing in Types.TIMESTAMP value instead of the original value. I think that we should fix that.

      1. SQOOP-906.4.txt
        14 kB
        Raghav Kumar Gautam
      2. SQOOP-906.txt
        11 kB
        Raghav Kumar Gautam
      3. SQOOP-906.txt
        13 kB
        Raghav Kumar Gautam
      4. SQOOP-906.txt
        5 kB
        Raghav Kumar Gautam

        Issue Links

          Activity

          Jarek Jarcec Cecho created issue -
          Raghav Kumar Gautam made changes -
          Field Original Value New Value
          Assignee Raghav Kumar Gautam [ raghavgautam ]
          Raghav Kumar Gautam made changes -
          Attachment SQOOP-906.txt [ 12589743 ]
          Raghav Kumar Gautam made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Jarek Jarcec Cecho made changes -
          Remote Link This issue links to "Review board (Web Link)" [ 12370 ]
          Hide
          Raghav Kumar Gautam added a comment -

          Updated patch.

          Show
          Raghav Kumar Gautam added a comment - Updated patch.
          Raghav Kumar Gautam made changes -
          Attachment SQOOP-906.txt [ 12590206 ]
          Raghav Kumar Gautam made changes -
          Attachment SQOOP-906.txt [ 12590292 ]
          Raghav Kumar Gautam made changes -
          Attachment SQOOP-906.4.txt [ 12592357 ]
          Hide
          ASF subversion and git services added a comment -

          Commit 5495113781286de00473eb3d8c535f4288454082 in branch refs/heads/trunk from Jarek Jarcec Cecho
          [ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=5495113 ]

          SQOOP-906: Sqoop is always calling ConnectionManager.datetimeToQueryString with TIMESTAMP column type

          (Raghav Kumar Gautam via Jarek Jarcec Cecho)

          Show
          ASF subversion and git services added a comment - Commit 5495113781286de00473eb3d8c535f4288454082 in branch refs/heads/trunk from Jarek Jarcec Cecho [ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=5495113 ] SQOOP-906 : Sqoop is always calling ConnectionManager.datetimeToQueryString with TIMESTAMP column type (Raghav Kumar Gautam via Jarek Jarcec Cecho)
          Hide
          Jarek Jarcec Cecho added a comment -

          Thank you Raghav Kumar Gautam for fixing this one!

          Show
          Jarek Jarcec Cecho added a comment - Thank you Raghav Kumar Gautam for fixing this one!
          Jarek Jarcec Cecho made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 1.4.4 [ 12324082 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop200 #817 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/817/)
          SQOOP-906: Sqoop is always calling ConnectionManager.datetimeToQueryString with TIMESTAMP column type (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=5495113781286de00473eb3d8c535f4288454082)

          • src/test/org/apache/sqoop/manager/oracle/OracleIncrementalImportTest.java
          • src/test/com/cloudera/sqoop/ThirdPartyTests.java
          • src/java/org/apache/sqoop/manager/ConnManager.java
          • src/java/org/apache/sqoop/tool/ImportTool.java
          • src/test/com/cloudera/sqoop/TestIncrementalImport.java
          • src/test/com/cloudera/sqoop/testutil/BaseSqoopTestCase.java
          • src/test/com/cloudera/sqoop/TestMerge.java
          • src/java/org/apache/sqoop/manager/OracleManager.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop200 #817 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/817/ ) SQOOP-906 : Sqoop is always calling ConnectionManager.datetimeToQueryString with TIMESTAMP column type (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=5495113781286de00473eb3d8c535f4288454082 ) src/test/org/apache/sqoop/manager/oracle/OracleIncrementalImportTest.java src/test/com/cloudera/sqoop/ThirdPartyTests.java src/java/org/apache/sqoop/manager/ConnManager.java src/java/org/apache/sqoop/tool/ImportTool.java src/test/com/cloudera/sqoop/TestIncrementalImport.java src/test/com/cloudera/sqoop/testutil/BaseSqoopTestCase.java src/test/com/cloudera/sqoop/TestMerge.java src/java/org/apache/sqoop/manager/OracleManager.java
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop100 #778 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/778/)
          SQOOP-906: Sqoop is always calling ConnectionManager.datetimeToQueryString with TIMESTAMP column type (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=5495113781286de00473eb3d8c535f4288454082)

          • src/java/org/apache/sqoop/manager/OracleManager.java
          • src/java/org/apache/sqoop/tool/ImportTool.java
          • src/test/com/cloudera/sqoop/TestMerge.java
          • src/test/org/apache/sqoop/manager/oracle/OracleIncrementalImportTest.java
          • src/test/com/cloudera/sqoop/testutil/BaseSqoopTestCase.java
          • src/test/com/cloudera/sqoop/ThirdPartyTests.java
          • src/test/com/cloudera/sqoop/TestIncrementalImport.java
          • src/java/org/apache/sqoop/manager/ConnManager.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop100 #778 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/778/ ) SQOOP-906 : Sqoop is always calling ConnectionManager.datetimeToQueryString with TIMESTAMP column type (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=5495113781286de00473eb3d8c535f4288454082 ) src/java/org/apache/sqoop/manager/OracleManager.java src/java/org/apache/sqoop/tool/ImportTool.java src/test/com/cloudera/sqoop/TestMerge.java src/test/org/apache/sqoop/manager/oracle/OracleIncrementalImportTest.java src/test/com/cloudera/sqoop/testutil/BaseSqoopTestCase.java src/test/com/cloudera/sqoop/ThirdPartyTests.java src/test/com/cloudera/sqoop/TestIncrementalImport.java src/java/org/apache/sqoop/manager/ConnManager.java
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop23 #1019 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/1019/)
          SQOOP-906: Sqoop is always calling ConnectionManager.datetimeToQueryString with TIMESTAMP column type (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=5495113781286de00473eb3d8c535f4288454082)

          • src/test/com/cloudera/sqoop/testutil/BaseSqoopTestCase.java
          • src/java/org/apache/sqoop/manager/ConnManager.java
          • src/test/com/cloudera/sqoop/ThirdPartyTests.java
          • src/test/com/cloudera/sqoop/TestIncrementalImport.java
          • src/java/org/apache/sqoop/manager/OracleManager.java
          • src/java/org/apache/sqoop/tool/ImportTool.java
          • src/test/org/apache/sqoop/manager/oracle/OracleIncrementalImportTest.java
          • src/test/com/cloudera/sqoop/TestMerge.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop23 #1019 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/1019/ ) SQOOP-906 : Sqoop is always calling ConnectionManager.datetimeToQueryString with TIMESTAMP column type (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=5495113781286de00473eb3d8c535f4288454082 ) src/test/com/cloudera/sqoop/testutil/BaseSqoopTestCase.java src/java/org/apache/sqoop/manager/ConnManager.java src/test/com/cloudera/sqoop/ThirdPartyTests.java src/test/com/cloudera/sqoop/TestIncrementalImport.java src/java/org/apache/sqoop/manager/OracleManager.java src/java/org/apache/sqoop/tool/ImportTool.java src/test/org/apache/sqoop/manager/oracle/OracleIncrementalImportTest.java src/test/com/cloudera/sqoop/TestMerge.java
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop20 #819 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/819/)
          SQOOP-906: Sqoop is always calling ConnectionManager.datetimeToQueryString with TIMESTAMP column type (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=5495113781286de00473eb3d8c535f4288454082)

          • src/test/com/cloudera/sqoop/ThirdPartyTests.java
          • src/test/com/cloudera/sqoop/testutil/BaseSqoopTestCase.java
          • src/test/com/cloudera/sqoop/TestMerge.java
          • src/java/org/apache/sqoop/manager/ConnManager.java
          • src/java/org/apache/sqoop/tool/ImportTool.java
          • src/test/com/cloudera/sqoop/TestIncrementalImport.java
          • src/java/org/apache/sqoop/manager/OracleManager.java
          • src/test/org/apache/sqoop/manager/oracle/OracleIncrementalImportTest.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop20 #819 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/819/ ) SQOOP-906 : Sqoop is always calling ConnectionManager.datetimeToQueryString with TIMESTAMP column type (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=5495113781286de00473eb3d8c535f4288454082 ) src/test/com/cloudera/sqoop/ThirdPartyTests.java src/test/com/cloudera/sqoop/testutil/BaseSqoopTestCase.java src/test/com/cloudera/sqoop/TestMerge.java src/java/org/apache/sqoop/manager/ConnManager.java src/java/org/apache/sqoop/tool/ImportTool.java src/test/com/cloudera/sqoop/TestIncrementalImport.java src/java/org/apache/sqoop/manager/OracleManager.java src/test/org/apache/sqoop/manager/oracle/OracleIncrementalImportTest.java

            People

            • Assignee:
              Raghav Kumar Gautam
              Reporter:
              Jarek Jarcec Cecho
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development