Uploaded image for project: 'Sqoop'
  1. Sqoop
  2. SQOOP-3061

Sqoop --options-file failed with error "Malformed option in options file" even though the query is correct

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.4.6
    • Fix Version/s: 1.4.7
    • Component/s: None
    • Labels:

      Description

      if you have the following in the options file:

      --query
      SELECT * FROM test WHERE a = 'b'

      and then run

      sqoop --options-file <file-path>
      

      it will fail with the following error:

      16/11/22 16:08:59 ERROR sqoop.Sqoop: Error while expanding arguments
      java.lang.Exception: Malformed option in options file(/tmp/sqoop_runner_from_stdin_1112_12354__sqoop_options_file): SELECT * FROM test WHERE a = 'b'
      at org.apache.sqoop.util.OptionsFileUtil.removeQuoteCharactersIfNecessary(OptionsFileUtil.java:170)
      at org.apache.sqoop.util.OptionsFileUtil.removeQuotesEncolosingOption(OptionsFileUtil.java:136)
      at org.apache.sqoop.util.OptionsFileUtil.expandArguments(OptionsFileUtil.java:90)
      at com.cloudera.sqoop.util.OptionsFileUtil.expandArguments(OptionsFileUtil.java:33)
      at org.apache.sqoop.Sqoop.runTool(Sqoop.java:199)
      at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
      at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
      Malformed option in options file(/tmp/sqoop_runner_from_stdin_1112_12354__sqoop_options_file): SELECT * FROM test WHERE a = 'b'
      

      This is caused by function org.apache.sqoop.util.OptionsFileUtil.removeQuoteCharactersIfNecessary only checks for starting and ending quotes and will fail if the query does not start with a quote but ends with a quote, like the example query above.

        private static String removeQuoteCharactersIfNecessary(String fileName,
            String option, char quote) throws Exception {
          boolean startingQuote = (option.charAt(0) == quote);
          boolean endingQuote = (option.charAt(option.length() - 1) == quote);
      
          if (startingQuote && endingQuote) {
            if (option.length() == 1) {
              throw new Exception("Malformed option in options file("
                  + fileName + "): " + option);
            }
            return option.substring(1, option.length() - 1);
          }
      
          if (startingQuote || endingQuote) {
             throw new Exception("Malformed option in options file("
                 + fileName + "): " + option);
          }
      
          return option;
        }
      

        Attachments

        1. SQOOP-3061.patch
          5 kB
          Eric Lin
        2. SQOOP-3061.5.patch
          6 kB
          Eric Lin
        3. SQOOP-3061.4.patch
          7 kB
          Eric Lin
        4. SQOOP-3061.3.patch
          5 kB
          Eric Lin
        5. SQOOP-3061.2.patch
          5 kB
          Eric Lin

          Activity

            People

            • Assignee:
              ericlin Eric Lin
              Reporter:
              ericlin Eric Lin
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: