[SQOOP-2639] Unable to export utf-8 data to MySQL using --direct mode

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.4.6
• Component/s: connectors/mysql
• Assignee: DaeMyung Kang
• Reporter: Ranjan Bagchi
• Created: 23/Oct/15 18:01
• Updated: 18/Sep/18 07:23

Description

I am able to import utf-8 data (non-latin1) data successfully into HDFS via:

sqoop import --connect jdbc:mysql://host/db --username XX --password YY \
--mysql-delimiters \
--table MYSQL_SRC_TABLE --target-dir ${SQOOP_DIR_PREFIX}/mysql_table --direct

However, using

sqoop export --connect jdbc:mysql://host/db --username XX --password YY \
--mysql-delimiters \
--table MYSQL_DEST_TABLE --export-dir ${SQOOP_DIR_PREFIX}/mysql_table \
--direct

Cuts off the fields after the first non-latin1 character (eg a letter w/ an umlaut).
I tried other options like – --default-character-set=utf8, without success.

I was able to fix the problem with the following change:
Change https://svn.apache.org/repos/asf/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/MySQLExportMapper.java, line 322 from
this.mysqlCharSet = MySQLUtils.MYSQL_DEFAULT_CHARSET;
to
this.mysqlCharSet = "utf-8";

Hope this helps