Description
1/ SPARK-33428 fixed ArrayIndexOutofBoundsException but introduced a new inconsistency in behaviour where the returned value is different above the 64 char threshold.
scala> spark.sql("select conv(repeat('?', 64), 10, 16)").show
+---------------------------+
|conv(repeat(?, 64), 10, 16)|
+---------------------------+
| 0|
+---------------------------+
scala> spark.sql("select conv(repeat('?', 65), 10, 16)").show
+---------------------------+
|conv(repeat(?, 65), 10, 16)|
+---------------------------+
| FFFFFFFFFFFFFFFF|
+---------------------------+
scala> spark.sql("select conv(repeat('?', 65), 10, -16)").show
+----------------------------+
|conv(repeat(?, 65), 10, -16)|
+----------------------------+
| -1|
+----------------------------+
scala> spark.sql("select conv(repeat('?', 64), 10, -16)").show
+----------------------------+
|conv(repeat(?, 64), 10, -16)|
+----------------------------+
| 0|
+----------------------------+
2/ conv should return result equal to max unsigned long value in base toBase when there is overflow
scala> spark.sql(select conv('aaaaaaa0aaaaaaa0a', 16, 10)).show // which should be 18446744073709551615 +-------------------------------+ |conv(aaaaaaa0aaaaaaa0a, 16, 10)| +-------------------------------+ | 12297828695278266890| +-------------------------------+
Attachments
Issue Links
- is caused by
-
SPARK-33428 Match the behavior of conv function to MySQL's
-
- Resolved
-
- links to
