Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-8457

Hudi MIT doesn't support the target table reference in extra MATCHED condition

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Critical
    • Resolution: Unresolved
    • None
    • 1.1.0
    • spark-sql
    • None

    Description

      Below MIT query fails with Hudi - 

      ```{}

      {{MERGE INTO hudi_table_mor_2_partition_columns t
      USING comprehensive_merge_source s
      ON t.id = s.id
      WHEN MATCHED
      AND s.operation = 'UPDATE_DEPT_MATCH'
      AND t.department = s.department
      THEN
      UPDATE SET *
      WHEN MATCHED
      AND s.operation = 'UPDATE_SALARY' THEN
      UPDATE SET * ;}}

      ```

      if we just remove `t.department = s.department` from above query it works. It also works if we just remove reference for target table from the same. `t.department = 'A'` doesn't work but {}`s.department = 'A'` works well.

       

      GIthub issue have reproducible code - https://github.com/apache/hudi/issues/12180

      Confirmed that DELTA supports these kind of MIT queries but Hudi fails with the Exception.  Also checked with master code and issue still exists.

      Attachments

        Activity

          People

            Unassigned Unassigned
            adityagoenka Aditya Goenka
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: