[SQOOP-1617] MySQL fetch-size behavior changed with SQOOP-1400

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.4.6
• Fix Version/s: 1.4.6
• Component/s: connectors/mysql

Environment

CDH 5.2
sqoop 1.4.5 (seems to include SQOOP-1400)
mysql connector version 5.1.33

• Assignee: Jarek Jarcec Cecho
• Reporter: Jürgen Thomann
• Created: 23/Oct/14 08:05
• Updated: 17/Nov/15 16:28
• Resolved: 11/Nov/14 01:07

Description

SQOOP-1400 changed the default behavior for the connector to load everything in memory. The only working way to get the old streaming back is to use --fetch-size -2147483648 (Integer.MIN_VALUE)

It would be nice if that could be changed and/or documented that mysql does not support a fetch size and does only support row-by-row or loading everything in memory.

The issue is discussed for example here:

http://community.cloudera.com/t5/Data-Ingestion-Integration/Sqoop-GC-overhead-limit-exceeded-after-CDH5-2-update/td-p/20604

Issue Links

• is related to: SQOOP-1400 Failed to import data using mysql-connector-java-5.1.17.jar