Uploaded image for project: 'Sqoop (Retired)'
  1. Sqoop (Retired)
  2. SQOOP-3356

SQL Syntax Error Raised when Double Quotes Added for "check-column"

Add voteWatch issue
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Blocker
    • Resolution: Unresolved
    • 1.4.6
    • None
    • connectors/oracle, metastore
    • None

    Description

      Goal: Implementing Sqoop Metastore to run daily incremental ETL.

      When executing the following Sqoop job:

      sqoop job -Dmapreduce.job.queuename=D_NO_SLA \
   --create yle0_gim_interaction_fact - import \
   --connect "jdbc:oracle:thin:@//xxxx.xxxx.com:0000/xxxx" \
   --username T0V0_DATALAKE_YLE0 \
   --password-file /dev/xxxxxxx/YLE0_OAT.password 
   --outdir ./sqoop/ \
   --table RYLEINFOMART_2.INTERACTION_FACT \
   --incremental append \
   --check-column 'GREATEST(CREATE_AUDIT_KEY, UPDATE_AUDIT_KEY)' \
   --last-value '53492217' \
   --split-by 'CREATE_AUDIT_KEY' -m 16 \
   --target-dir /dev/T0V0/hivedb/yle0/staging/yle0_gim_interaction_fact_staging/ \
   --fields-terminated-by '\001'

      Sqoop would automatically run the following query to update the max value:

      SELECT MAX("GREATEST(CREATE_AUDIT_KEY, UPDATE_AUDIT_KEY)") FROM "RYLEINFOMART_2"."INTERACTION_FACT"

      SQL syntax error was given since there shouldn't be double quotes around GREATEST() function.

      The proposal of using free form query is rejected since metastore cannot be implemented to automatically run the daily incremental job.

      Attachments

        Activity

          People

            Unassigned Unassigned
            seanmzx Sean Meng
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:

              Slack

                Issue deployment