[SPARK-30633] Codegen fails when xxHash seed is not an integer

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.4.4
• Fix Version/s: 2.4.5, 3.0.0
• Component/s: SQL
• Assignee: Patrick Cording
• Reporter: Patrick Cording
• Created: 24/Jan/20 10:47
• Updated: 27/Jan/20 19:22
• Resolved: 27/Jan/20 18:33

Description

If the seed for xxHash is not an integer the generated code does not compile.

Steps to reproduce:

import org.apache.spark.sql.catalyst.expressions.XxHash64
import org.apache.spark.sql.Column

val file = "..."
val column = col("...")

val df = spark.read.csv(file)

def xxHash(seed: Long, cols: Column*): Column = new Column(
   XxHash64(cols.map(_.expr), seed)
)

val seed = (Math.pow(2, 32)+1).toLong
df.select(xxHash(seed, column)).show()

Appending an L to the seed when the datatype is long fixes the issue.

Issue Links

• links to: GitHub Pull Request #27354