Uploaded image for project: 'Sqoop'
  1. Sqoop
  2. SQOOP-1329

JDBC connection to Oracle timeout after data import but before hive metadata import

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 1.4.4
    • Fix Version/s: 1.4.5
    • Component/s: connectors/oracle
    • Labels:
    • Environment:

      Red Hat Enterprise Linux Server release 6.5

      Description

      When I'm importing a table from Oracle which takes more than 1 hour to extract, I'm getting following error message at the stage when Sqoop tries to import data from temporary HDFS location to Hive:

      14/05/27 13:05:51 INFO mapreduce.ImportJobBase: Transferred 47.2606 GB in 6,389.4644 seconds (6.7206 MB/sec)
      14/05/27 13:05:51 INFO mapreduce.ImportJobBase: Retrieved 98235461 records.
      14/05/27 13:05:51 DEBUG util.ClassLoaderStack: Restoring classloader: sun.misc.Launcher$AppClassLoader@566d0085
      14/05/27 13:05:51 DEBUG hive.HiveImport: Hive.inputTable: WAREHOUSE.MY_BIG_TABLE
      14/05/27 13:05:51 DEBUG hive.HiveImport: Hive.outputTable: WAREHOUSE.MY_BIG_TABLE
      14/05/27 13:05:51 DEBUG manager.OracleManager: Using column names query: SELECT t.* FROM WAREHOUSE.MY_BIG_TABLE t WHERE 1=0
      14/05/27 13:05:51 DEBUG manager.SqlManager: Execute getColumnTypesRawQuery : SELECT t.* FROM WAREHOUSE.MY_BIG_TABLE t WHERE 1=0
      14/05/27 13:05:51 ERROR manager.SqlManager: Error executing statement: java.sql.SQLException: ORA-02396: exceeded maximum idle time, please connect again

      java.sql.SQLException: ORA-02396: exceeded maximum idle time, please connect again

      at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
      at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:389)

      With small tables (under 1 hour) everything is fine.

      I'm using Sqoop v1.4.4

      14/05/27 13:49:14 INFO sqoop.Sqoop: Running Sqoop version: 1.4.4-cdh5.0.0
      Sqoop 1.4.4-cdh5.0.0
      git commit id 8e266e052e423af592871e2dfe09d54c03f6a0e8

      This problems looks exactly as described in (SQOOP-934) issue.

        Attachments

        1. SQOOP-1329.patch
          0.8 kB
          Andrey Dmitriev

          Activity

            People

            • Assignee:
              biosetup Andrey Dmitriev
              Reporter:
              biosetup Andrey Dmitriev
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: