[SQOOP-330] Free form query import with column transformation failed without obvious error message

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.4.0-incubating
• Fix Version/s: 1.4.0-incubating
• Component/s: tools
• Assignee: Jarek Jarcec Cecho
• Reporter: Jarek Jarcec Cecho
• Created: 04/Sep/11 17:43
• Updated: 07/Sep/11 03:10
• Resolved: 07/Sep/11 02:46

Description

Calling upstream version with free form query that contains column with transformation will fail on MS SQL server (I did not try other RDBMS):

[root@hddev01 sqoop]# sqoop import --driver com.microsoft.sqlserver.jdbc.SQLServerDriver --connect jdbc:sqlserver://host\;databaseName=db --username jarcec --password password --as-textfile --num-mappers 10 --target-dir /user/root/table SELECT convert(varchar(50), md5, 2) FROM table where $CONDITIONS' --split-by id
11/09/04 13:16:41 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
11/09/04 13:16:41 INFO manager.SqlManager: Using default fetchSize of 1000
11/09/04 13:16:41 INFO tool.CodeGenTool: Beginning code generation
11/09/04 13:16:43 INFO manager.SqlManager: Executing SQL statement: SELECT convert(varchar(50), md5, 2) FROM table where (1 = 0)
11/09/04 13:16:43 INFO manager.SqlManager: Executing SQL statement: SELECT convert(varchar(50), md5, 2) FROM table where (1 = 0)
11/09/04 13:16:43 INFO orm.CompilationManager: HADOOP_HOME is /usr/lib/hadoop
11/09/04 13:16:43 INFO orm.CompilationManager: Found hadoop core jar at: /usr/lib/hadoop/hadoop-core.jar
/tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:37: <identifier> expected
private String ;
^
/tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:41: <identifier> expected
public void set_(String ) {
^
/tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:42: <identifier> expected
this. = ;
^
/tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:42: illegal start of expression
this. = ;
^
/tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:44: <identifier> expected
public QueryResult with_(String ) {
^
/tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:45: <identifier> expected
this. = ;
^
/tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:45: illegal start of expression
this. = ;
^
/tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:57: <identifier> expected
equal = equal && (this. == null ? that. == null : this..equals(that.));
^
/tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:57: <identifier> expected
equal = equal && (this. == null ? that. == null : this..equals(that.));
^
/tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:57: malformed floating point literal
equal = equal && (this. == null ? that. == null : this..equals(that.));
^
/tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:57: <identifier> expected
equal = equal && (this. == null ? that. == null : this..equals(that.));
^
/tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:57: ';' expected
equal = equal && (this. == null ? that. == null : this..equals(that.));
^
/tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:62: <identifier> expected
this. = JdbcWritableBridge.readString(1, __dbResults);
^
/tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:72: illegal start of expression
JdbcWritableBridge.writeString(, 1 + __off, 12, __dbStmt);
^
/tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:77: <identifier> expected
this. = null;
^
/tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:79: <identifier> expected
this. = Text.readString(__dataIn);
^
/tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:83: <identifier> expected
if (null == this.) {
^
/tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:87: illegal start of expression
Text.writeString(__dataOut, );
^
/tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:103: illegal start of expression
__sb.append(FieldFormatter.escapeAndEnclose(==null?"null":, delimiters));
^
/tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:103: illegal start of expression
__sb.append(FieldFormatter.escapeAndEnclose(==null?"null":, delimiters));
^
/tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:163: <identifier> expected
if (__cur_str.equals("null"))

{ this. = null; }

else {
^
/tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:164: <identifier> expected
this. = __cur_str;
^
/tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:176: <identifier> expected
__sqoop$field_map.put("", this.);
^
/tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:182: <identifier> expected
this. = (String) __fieldVal;
^
24 errors
11/09/04 13:16:43 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Error returned by javac
at com.cloudera.sqoop.orm.CompilationManager.compile(CompilationManager.java:205)
at com.cloudera.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:83)
at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:367)
at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:453)
at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180)
at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:219)
at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:228)
at com.cloudera.sqoop.Sqoop.main(Sqoop.java:237)