[SQOOP-3061] Sqoop --options-file failed with error "Malformed option in options file" even though the query is correct

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.4.6
• Fix Version/s: 1.4.7
• Labels: patch
• Assignee: Eric Lin
• Reporter: Eric Lin
• Created: 25/Nov/16 04:24
• Updated: 16/Jun/17 04:30
• Resolved: 15/Feb/17 14:19

Description

if you have the following in the options file:

--query
SELECT * FROM test WHERE a = 'b'

and then run

sqoop --options-file <file-path>

it will fail with the following error:

16/11/22 16:08:59 ERROR sqoop.Sqoop: Error while expanding arguments
java.lang.Exception: Malformed option in options file(/tmp/sqoop_runner_from_stdin_1112_12354__sqoop_options_file): SELECT * FROM test WHERE a = 'b'
at org.apache.sqoop.util.OptionsFileUtil.removeQuoteCharactersIfNecessary(OptionsFileUtil.java:170)
at org.apache.sqoop.util.OptionsFileUtil.removeQuotesEncolosingOption(OptionsFileUtil.java:136)
at org.apache.sqoop.util.OptionsFileUtil.expandArguments(OptionsFileUtil.java:90)
at com.cloudera.sqoop.util.OptionsFileUtil.expandArguments(OptionsFileUtil.java:33)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:199)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
Malformed option in options file(/tmp/sqoop_runner_from_stdin_1112_12354__sqoop_options_file): SELECT * FROM test WHERE a = 'b'

This is caused by function org.apache.sqoop.util.OptionsFileUtil.removeQuoteCharactersIfNecessary only checks for starting and ending quotes and will fail if the query does not start with a quote but ends with a quote, like the example query above.

  private static String removeQuoteCharactersIfNecessary(String fileName,
      String option, char quote) throws Exception {
    boolean startingQuote = (option.charAt(0) == quote);
    boolean endingQuote = (option.charAt(option.length() - 1) == quote);

    if (startingQuote && endingQuote) {
      if (option.length() == 1) {
        throw new Exception("Malformed option in options file("
            + fileName + "): " + option);
      }
      return option.substring(1, option.length() - 1);
    }

    if (startingQuote || endingQuote) {
       throw new Exception("Malformed option in options file("
           + fileName + "): " + option);
    }

    return option;
  }