[SQOOP-446] sqoop column terminator "fields-terminated-by" needs to work with any of the following '\007', '0x7', '^G'

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.3.0
• Fix Version/s: 1.4.1-incubating
• Component/s: tools

Environment

currently using sqoop 1.3.x our data is UTF8 and since we have columns with most of the ascii keyboard char, it would be best to use the Bell char '\007'

• Assignee: Unassigned
• Reporter: mark monroe
• Created: 24/Feb/12 18:12
• Updated: 04/Apr/12 19:38
• Resolved: 04/Apr/12 19:38

Description

here is the command

sqoop export -conf /opt/hadoop/sqoop/sqoop-pdw-1.0/conf/pdwp.xml -libjars $SQOOP_HOME/lib/commons-net-2.2.jar -Dmapreduce.task.classpath.user.precedence=true --connect pdw://10.230.25.1 --username ETL_TEST --password ***** --table OPS_TEST_DB.dbo.PROTO_TEST_FAILING_PINS_sqoop -m 1 --export-dir /tmp/mm_test3 --input-fields-terminated-by "\007"

here is the error....

12/02/24 12:11:43 ERROR tool.ExportTool: Encountered IOException running export job: org.apache.hadoop.ipc.RemoteException: java.io.IOException: java.lang.RuntimeException: org.xml.sax.SAXParseException: Character reference "&#7" is an invalid XML character.
at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3766)
at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
Caused by: java.lang.RuntimeException: org.xml.sax.SAXParseException: Character reference "&#7" is an invalid XML character.
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1393)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1261)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1192)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:415)
at org.apache.hadoop.mapred.JobConf.checkAndWarnDeprecation(JobConf.java:1957)
at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:386)
at org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:414)
at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3764)
... 10 more
Caused by: org.xml.sax.SAXParseException: Character reference "&#7" is an invalid XML character.
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:249)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:124)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1313)
... 17 more