Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-49476

Base64 nullability change

    XMLWordPrintableJSON

Details

    Description

      In https://github.com/apache/spark/pull/47303 , the Base64 function was updated to use `StaticInvoke` instead of directly calling the underlying encode function. This change also updated Base64 of a non-nullable child to be nullable instead of non-nullable. This was backported to Spark 3.5 as well (https://github.com/apache/spark/pull/47325) and has caused issues with failing state schema checks for stateful streaming jobs. This should be fixed to have the nullability based on the child.

      Attachments

        Activity

          People

            kimahriman Adam Binford
            kimahriman Adam Binford
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: