Uploaded image for project: 'Sqoop'
  1. Sqoop
  2. SQOOP-1251

Sqoop job performing Incremental import hangs after the successful import

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.4.3
    • Fix Version/s: None
    • Component/s: metastore
    • Labels:
      None

      Description

      To reproduce the problem, create an incremental import job.
      $SQOOP_HOME/bin/sqoop job --create mergetest – import --options-file sqoop-params.txt --table mergetest --incremental lastmodified --check-column lastmod -m 1
      In subsequent "exec", the import completed successfully but the process hung.

      It appears that hsqldb v1.8 used in the metastore only allows "a single session at a time performs an SQL statement completely, before the next session is allowed access." In this scenario, the JobTool created a session to retrieve the previously saved Job info and the ImportTool created a separate connection for saving incremental info. Adding a connection.rollback() call in the read(String jobName) method of HsqldbJobStorage.java eliminated the hang. Upgrading hsqldb to v2.0 might also resolve the problem.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              lmau Louis Mau
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: