Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-3857

Add parameter to change sql quoting character for sqoop

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • v3.0.0, v2.6.2
    • None
    • None

    Description

      I've set up  Kylin to access a SQL server using JDBC and do the ETL step as described in

      http://kylin.apache.org/docs/tutorial/setup_jdbc_datasource.html

      but when the "Sqoop To Flat Hive Table" runs it terminates with an error shown in the attached log.

      It seems that the root cause is the presence of the '`' character in the query, which has no meaning for sql server.

       

      My problem seems to related to this piece of code

       

      public class FlatTableSqlQuoteUtils {

      public static final String QUOTE = "`";

      /**

      • Quote identifier by default quote `
      • @param identifier
      • @return
        */
        public static String quoteIdentifier(String identifier) { return QUOTE + identifier + QUOTE; }

       

      The solution may be to add a parameter to change this character used for quoting.

       

      Just to be sure I made a copy of the original "sqoop" command to "sqoop.orig" and replaced the original with a little script of mine (attached).

      This little script removes the "`" character from the original command and the call "sqoop.orig" with the cleaned arguments.

      It works, but it is still an ugly workaround.

       

      Attachments

        1. image-2019-03-11-11-46-05-946.png
          59 kB
          Xiaoxiang Yu
        2. image-2019-03-11-11-46-14-305.png
          101 kB
          Xiaoxiang Yu
        3. image-2019-03-11-11-46-20-664.png
          175 kB
          Xiaoxiang Yu
        4. image-2019-03-11-11-46-31-966.png
          200 kB
          Xiaoxiang Yu
        5. sqoop-error.log
          8 kB
          Davide Malagoli
        6. sqoop-script.sh
          2 kB
          Davide Malagoli

        Issue Links

          Activity

            People

              hit_lacus Xiaoxiang Yu
              darkice01 Davide Malagoli
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: