Sqoop
  1. Sqoop
  2. SQOOP-330

Free form query import with column transformation failed without obvious error message

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.0-incubating
    • Fix Version/s: 1.4.0-incubating
    • Component/s: tools
    • Labels:
      None

      Description

      Calling upstream version with free form query that contains column with transformation will fail on MS SQL server (I did not try other RDBMS):

      [root@hddev01 sqoop]# sqoop import --driver com.microsoft.sqlserver.jdbc.SQLServerDriver --connect jdbc:sqlserver://host\;databaseName=db --username jarcec --password password --as-textfile --num-mappers 10 --target-dir /user/root/table SELECT convert(varchar(50), md5, 2) FROM table where $CONDITIONS' --split-by id
      11/09/04 13:16:41 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
      11/09/04 13:16:41 INFO manager.SqlManager: Using default fetchSize of 1000
      11/09/04 13:16:41 INFO tool.CodeGenTool: Beginning code generation
      11/09/04 13:16:43 INFO manager.SqlManager: Executing SQL statement: SELECT convert(varchar(50), md5, 2) FROM table where (1 = 0)
      11/09/04 13:16:43 INFO manager.SqlManager: Executing SQL statement: SELECT convert(varchar(50), md5, 2) FROM table where (1 = 0)
      11/09/04 13:16:43 INFO orm.CompilationManager: HADOOP_HOME is /usr/lib/hadoop
      11/09/04 13:16:43 INFO orm.CompilationManager: Found hadoop core jar at: /usr/lib/hadoop/hadoop-core.jar
      /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:37: <identifier> expected
      private String ;
      ^
      /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:41: <identifier> expected
      public void set_(String ) {
      ^
      /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:42: <identifier> expected
      this. = ;
      ^
      /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:42: illegal start of expression
      this. = ;
      ^
      /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:44: <identifier> expected
      public QueryResult with_(String ) {
      ^
      /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:45: <identifier> expected
      this. = ;
      ^
      /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:45: illegal start of expression
      this. = ;
      ^
      /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:57: <identifier> expected
      equal = equal && (this. == null ? that. == null : this..equals(that.));
      ^
      /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:57: <identifier> expected
      equal = equal && (this. == null ? that. == null : this..equals(that.));
      ^
      /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:57: malformed floating point literal
      equal = equal && (this. == null ? that. == null : this..equals(that.));
      ^
      /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:57: <identifier> expected
      equal = equal && (this. == null ? that. == null : this..equals(that.));
      ^
      /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:57: ';' expected
      equal = equal && (this. == null ? that. == null : this..equals(that.));
      ^
      /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:62: <identifier> expected
      this. = JdbcWritableBridge.readString(1, __dbResults);
      ^
      /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:72: illegal start of expression
      JdbcWritableBridge.writeString(, 1 + __off, 12, __dbStmt);
      ^
      /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:77: <identifier> expected
      this. = null;
      ^
      /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:79: <identifier> expected
      this. = Text.readString(__dataIn);
      ^
      /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:83: <identifier> expected
      if (null == this.) {
      ^
      /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:87: illegal start of expression
      Text.writeString(__dataOut, );
      ^
      /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:103: illegal start of expression
      __sb.append(FieldFormatter.escapeAndEnclose(==null?"null":, delimiters));
      ^
      /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:103: illegal start of expression
      __sb.append(FieldFormatter.escapeAndEnclose(==null?"null":, delimiters));
      ^
      /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:163: <identifier> expected
      if (__cur_str.equals("null"))

      { this. = null; }

      else {
      ^
      /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:164: <identifier> expected
      this. = __cur_str;
      ^
      /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:176: <identifier> expected
      __sqoop$field_map.put("", this.);
      ^
      /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:182: <identifier> expected
      this. = (String) __fieldVal;
      ^
      24 errors
      11/09/04 13:16:43 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Error returned by javac
      at com.cloudera.sqoop.orm.CompilationManager.compile(CompilationManager.java:205)
      at com.cloudera.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:83)
      at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:367)
      at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:453)
      at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
      at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180)
      at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:219)
      at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:228)
      at com.cloudera.sqoop.Sqoop.main(Sqoop.java:237)

      1. SQOOP-330.patch
        1 kB
        Jarek Jarcec Cecho
      2. SQOOP-330.patch
        1 kB
        Jarek Jarcec Cecho

        Activity

        Jarek Jarcec Cecho created issue -
        Jarek Jarcec Cecho made changes -
        Field Original Value New Value
        Attachment SQOOP-330.patch [ 12492983 ]
        Jarek Jarcec Cecho made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Arvind Prabhakar made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Jarek Jarcec Cecho made changes -
        Attachment SQOOP-330.patch [ 12493194 ]
        Arvind Prabhakar made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Fix Version/s 1.4.0 [ 12317345 ]
        Resolution Fixed [ 1 ]

          People

          • Assignee:
            Jarek Jarcec Cecho
            Reporter:
            Jarek Jarcec Cecho
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development