Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-5324

Spark SQL MERGE INTO statement should always do upsert if there's matching update clause

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Critical
    • Resolution: Unresolved
    • None
    • None
    • index, spark-sql
    • None

    Description

      UPDATED

      Aforementioned issue was actually a result of misconfiguration of the Merge Into statement – MIT was using "insert" operation instead of "upsert".

      Real issue though is that MIT implicitly predicates using "upsert" operation onto whether "preCombine" config is set. Instead, it should always specify operation as "upsert", since MIT allows to specify updating semantics w/o requiring presence of the "preCombine" field

      Attachments

        Activity

          People

            alexey.kudinkin Alexey Kudinkin
            guoyihua Ethan Guo
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: