Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-7208

Do writing stage should shutdown with error when insert failed to reduce user execute time and show error details

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • spark-sql

    Description

      when user execute merge into or insert into to a table,if source table data is not right(such as source schema is not consistent to target table),should shutdown and throw error details to user,this can reduce cost of job and indicate more message for user to dig questions,

      job is not error evenwhen data insert occur error,this would misstake user and cost more compute time which is not necessary,stage status show success,but had been failed due to writing failed,like the pictures:

      Attachments

        1. CAPTURE_2023-12-11_144823.jpg
          125 kB
          xy
        2. CAPTURE_2023-12-11_144839.jpg
          78 kB
          xy
        3. CAPTURE_2023-12-11_144849.jpg
          46 kB
          xy
        4. CAPTURE_2023-12-11_144901.jpg
          49 kB
          xy

        Issue Links

          Activity

            People

              Unassigned Unassigned
              xuzifu xy
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated: