[SQOOP-1251] Sqoop job performing Incremental import hangs after the successful import

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.4.3
• Component/s: metastore
• Assignee: Unassigned
• Reporter: Louis Mau
• Created: 05/Dec/13 18:31
• Updated: 05/Dec/13 18:31

Description

To reproduce the problem, create an incremental import job.
$SQOOP_HOME/bin/sqoop job --create mergetest – import --options-file sqoop-params.txt --table mergetest --incremental lastmodified --check-column lastmod -m 1
In subsequent "exec", the import completed successfully but the process hung.

It appears that hsqldb v1.8 used in the metastore only allows "a single session at a time performs an SQL statement completely, before the next session is allowed access." In this scenario, the JobTool created a session to retrieve the previously saved Job info and the ImportTool created a separate connection for saving incremental info. Adding a connection.rollback() call in the read(String jobName) method of HsqldbJobStorage.java eliminated the hang. Upgrading hsqldb to v2.0 might also resolve the problem.