[SPARK-37858] Throw Spark exceptions from AES functions

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.0
• Fix Version/s: 3.3.0
• Component/s: SQL
• Assignee: Max Gekk
• Reporter: Max Gekk
• Created: 10/Jan/22 19:24
• Updated: 07/Mar/23 13:07
• Resolved: 12/Jan/22 14:39

Description

Currently, Spark SQL can throw Java exceptions from the aes_encrypt()/aes_decrypt() functions, for instance:

java.lang.RuntimeException: javax.crypto.AEADBadTagException: Tag mismatch!
	at org.apache.spark.sql.catalyst.expressions.ExpressionImplUtils.aesInternal(ExpressionImplUtils.java:93)
	at org.apache.spark.sql.catalyst.expressions.ExpressionImplUtils.aesDecrypt(ExpressionImplUtils.java:43)
	at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.project_doConsume_0$(Unknown Source)
	at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown Source)
	at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
	at org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$1.hasNext(WholeStageCodegenExec.scala:759)
	at org.apache.spark.sql.execution.SparkPlan.$anonfun$getByteArrayRdd$1(SparkPlan.scala:354)
	at org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2(RDD.scala:890)
	at org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2$adapted(RDD.scala:890)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:365)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:329)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
	at org.apache.spark.scheduler.Task.run(Task.scala:136)
	at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:507)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1468)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:510)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: javax.crypto.AEADBadTagException: Tag mismatch!
	at com.sun.crypto.provider.GaloisCounterMode.decryptFinal(GaloisCounterMode.java:620)
	at com.sun.crypto.provider.CipherCore.finalNoPadding(CipherCore.java:1116)
	at com.sun.crypto.provider.CipherCore.fillOutputBuffer(CipherCore.java:1053)
	at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:853)
	at com.sun.crypto.provider.AESCipher.engineDoFinal(AESCipher.java:446)
	at javax.crypto.Cipher.doFinal(Cipher.java:2226)
	at org.apache.spark.sql.catalyst.expressions.ExpressionImplUtils.aesInternal(ExpressionImplUtils.java:87)
	... 19 more

That might confuse non-Scala/Java users. Need to wrap such kind of exception by Spark's exception.

Issue Links

• is caused by: SPARK-12567 Add aes_encrypt and aes_decrypt UDFs
• is cloned by: SPARK-38481 Substitute Java overflow exception from TIMESTAMPADD by Spark exception
• is related to: SPARK-42701 Add the try_aes_decrypt() function
• relates to: SPARK-37586 Add cipher mode option and set default cipher mode for aes_encrypt and aes_decrypt
• relates to: SPARK-37591 Support the GCM mode by aes_encrypt()/aes_decrypt()
• relates to: SPARK-37666 Set `GCM` as the default mode in `aes_encrypt()`/`aes_decrypt()`
• links to: [Github] Pull Request #35157 (MaxGekk)
• links to: [Github] Pull Request #35328 (imback82)
• links to: [Github] Pull Request #35328 (imback82)